-- Logs begin at Thu 2022-09-01 16:47:58 UTC, end at Thu 2022-09-01 17:59:45 UTC. -- Sep 01 17:16:42.741813 dv-cinder-852211-8-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 01 17:16:42.749601 dv-cinder-852211-8-iscsi devstack@keystone.service[81485]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 01 17:16:42.752432 dv-cinder-852211-8-iscsi devstack@keystone.service[81485]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Sep 1 17:16:42 2022] *** Sep 01 17:16:42.753101 dv-cinder-852211-8-iscsi devstack@keystone.service[81485]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 01 17:16:42.753242 dv-cinder-852211-8-iscsi devstack@keystone.service[81485]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 01 17:16:42.753409 dv-cinder-852211-8-iscsi devstack@keystone.service[81485]: nodename: dv-cinder-852211-8-iscsi Sep 01 17:16:42.753547 dv-cinder-852211-8-iscsi devstack@keystone.service[81485]: machine: x86_64 Sep 01 17:16:42.753681 dv-cinder-852211-8-iscsi devstack@keystone.service[81485]: clock source: unix Sep 01 17:16:42.753811 dv-cinder-852211-8-iscsi devstack@keystone.service[81485]: pcre jit disabled Sep 01 17:16:42.753944 dv-cinder-852211-8-iscsi devstack@keystone.service[81485]: detected number of CPU cores: 4 Sep 01 17:16:42.754076 dv-cinder-852211-8-iscsi devstack@keystone.service[81485]: current working directory: / Sep 01 17:16:42.754202 dv-cinder-852211-8-iscsi devstack@keystone.service[81485]: detected binary path: /usr/bin/uwsgi-core Sep 01 17:16:42.754343 dv-cinder-852211-8-iscsi devstack@keystone.service[81485]: your processes number limit is 47680 Sep 01 17:16:42.754478 dv-cinder-852211-8-iscsi devstack@keystone.service[81485]: your memory page size is 4096 bytes Sep 01 17:16:42.754614 dv-cinder-852211-8-iscsi devstack@keystone.service[81485]: detected max file descriptor number: 2048 Sep 01 17:16:42.754750 dv-cinder-852211-8-iscsi devstack@keystone.service[81485]: lock engine: pthread robust mutexes Sep 01 17:16:42.755041 dv-cinder-852211-8-iscsi devstack@keystone.service[81485]: thunder lock: enabled Sep 01 17:16:42.755362 dv-cinder-852211-8-iscsi devstack@keystone.service[81485]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 01 17:16:42.755574 dv-cinder-852211-8-iscsi devstack@keystone.service[81485]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 01 17:16:42.793858 dv-cinder-852211-8-iscsi devstack@keystone.service[81485]: Python main interpreter initialized at 0x56517b8a99c0 Sep 01 17:16:42.793858 dv-cinder-852211-8-iscsi devstack@keystone.service[81485]: python threads support enabled Sep 01 17:16:42.793858 dv-cinder-852211-8-iscsi devstack@keystone.service[81485]: your server socket listen backlog is limited to 100 connections Sep 01 17:16:42.793858 dv-cinder-852211-8-iscsi devstack@keystone.service[81485]: your mercy for graceful operations on workers is 90 seconds Sep 01 17:16:42.794224 dv-cinder-852211-8-iscsi devstack@keystone.service[81485]: mapped 403077 bytes (393 KB) for 2 cores Sep 01 17:16:42.794313 dv-cinder-852211-8-iscsi devstack@keystone.service[81485]: *** Operational MODE: preforking *** Sep 01 17:16:42.794551 dv-cinder-852211-8-iscsi devstack@keystone.service[81485]: *** uWSGI is running in multiple interpreter mode *** Sep 01 17:16:42.794551 dv-cinder-852211-8-iscsi devstack@keystone.service[81485]: spawned uWSGI master process (pid: 81485) Sep 01 17:16:42.794760 dv-cinder-852211-8-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 01 17:16:42.795313 dv-cinder-852211-8-iscsi devstack@keystone.service[81485]: spawned uWSGI worker 1 (pid: 81501, cores: 1) Sep 01 17:16:42.795646 dv-cinder-852211-8-iscsi devstack@keystone.service[81485]: spawned uWSGI worker 2 (pid: 81502, cores: 1) Sep 01 17:16:42.795795 dv-cinder-852211-8-iscsi devstack@keystone.service[81485]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 01 17:16:47.090446 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 01 17:16:47.090446 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: from cryptography.utils import int_from_bytes Sep 01 17:16:47.091214 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 01 17:16:47.091214 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: from cryptography.utils import int_from_bytes Sep 01 17:16:47.091335 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 01 17:16:47.091335 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: from cryptography.utils import int_from_bytes Sep 01 17:16:47.091740 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 01 17:16:47.091740 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: from cryptography.utils import int_from_bytes Sep 01 17:16:47.141170 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 01 17:16:47.141393 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 01 17:16:47.141491 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 01 17:16:47.141580 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 01 17:16:47.141658 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] command line args: [] {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 01 17:16:47.141734 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] command line args: [] {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 01 17:16:47.141809 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 01 17:16:47.141890 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 01 17:16:47.141966 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ================================================================================ {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 01 17:16:47.142042 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ================================================================================ {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 01 17:16:47.142204 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] admin_token = **** {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:16:47.142290 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] admin_token = **** {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:16:47.142443 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] config_dir = [] {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:16:47.142530 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] config_dir = [] {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:16:47.142628 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:16:47.142713 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:16:47.142810 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] config_source = [] {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:16:47.142894 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] config_source = [] {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:16:47.142973 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] debug = True {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:16:47.143045 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] debug = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:16:47.143202 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:16:47.143555 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:16:47.143691 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:16:47.143795 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:16:47.143898 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:16:47.143999 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:16:47.144100 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:16:47.144208 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:16:47.144311 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:16:47.144412 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:16:47.144507 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] list_limit = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:16:47.144606 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] list_limit = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:16:47.144712 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] log_config_append = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:16:47.144812 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] log_config_append = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:16:47.144812 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:16:47.144999 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:16:47.145098 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] log_dir = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:16:47.145190 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] log_dir = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:16:47.145281 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] log_file = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:16:47.145424 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] log_file = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:16:47.145543 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:16:47.145683 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:16:47.145779 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:16:47.145863 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:16:47.145979 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:16:47.146059 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:16:47.146183 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:16:47.146262 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:16:47.146379 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:16:47.146460 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:16:47.146531 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:16:47.146611 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:16:47.146682 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:16:47.146802 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:16:47.146882 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:16:47.146965 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:16:47.147041 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:16:47.147164 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:16:47.147245 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:16:47.147321 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:16:47.147479 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:16:47.147568 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:16:47.147712 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:16:47.147807 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:16:47.147956 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:16:47.148086 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:16:47.148196 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:16:47.148272 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:16:47.148468 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:16:47.148558 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:16:47.148955 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.102/identity {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:16:47.149032 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.102/identity {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:16:47.149132 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] publish_errors = False {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:16:47.149210 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] publish_errors = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:16:47.149386 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:16:47.149467 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:16:47.149616 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:16:47.149705 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:16:47.149782 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:16:47.149897 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:16:47.150021 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:16:47.150115 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:16:47.150253 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:16:47.150331 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:16:47.150484 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] standard_threads = False {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:16:47.150565 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] standard_threads = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:16:47.150711 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:16:47.150800 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:16:47.150948 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:16:47.151026 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:16:47.151121 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:16:47.151198 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:16:47.151296 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] use_journal = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:16:47.151422 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] use_journal = False {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:16:47.151496 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] use_json = False {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:16:47.151566 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] use_json = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:16:47.151634 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] use_stderr = False {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:16:47.151702 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] use_stderr = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:16:47.151776 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] use_syslog = False {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:16:47.151855 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] use_syslog = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:16:47.151996 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:16:47.152073 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:16:47.152246 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.152335 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.152481 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.152569 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.152741 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.152817 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.152973 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.153055 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.153217 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.153323 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.153477 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.153561 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.153721 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.153803 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.153953 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.154041 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.154197 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.154274 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.154439 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.154515 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.154684 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.154769 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.154916 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.154993 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.155158 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.155241 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.155392 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.155479 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.155632 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.155714 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.155875 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.155953 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.156109 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.156195 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.156362 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.156443 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.156611 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.156687 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.156852 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.157003 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.157082 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.157153 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.157340 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.157456 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.157600 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.157683 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.157852 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.157929 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.158095 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.158171 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.158325 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.158403 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.158550 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.158627 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.159291 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.159385 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.159542 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.159622 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.159794 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.159872 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.160042 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.160118 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.160273 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.160404 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.160504 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.160588 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.160749 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.160826 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.160991 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.161068 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.161225 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.161301 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.161494 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.161574 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.161723 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.161808 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.161955 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.162030 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.162187 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.162263 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.162414 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.162498 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.162641 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.162717 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.162866 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.162941 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.163085 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.163159 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.163311 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.163390 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.163556 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.163634 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.163783 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.163868 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.164017 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.164093 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.164244 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.164320 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.164475 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.164554 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.164705 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.164781 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.164951 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.165026 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.165176 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.165280 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.165433 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.165541 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.165638 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.165732 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.165875 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.165981 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.166109 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.166249 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.166336 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.166418 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.166541 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.166616 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.166789 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.166870 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.167020 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.167109 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.167254 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.167334 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.167479 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.167564 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.167734 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.167821 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.167967 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.168057 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.168174 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.168248 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.168411 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.168492 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.168642 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.168773 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.168907 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.168987 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.169139 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.169512 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.169589 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.169714 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.169830 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.169904 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.170017 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.170092 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.170206 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.170288 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.170387 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.170462 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.170532 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.170719 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.170837 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.170920 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.170993 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.171063 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.171186 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.171264 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.171420 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.171506 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.171663 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.171741 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.171897 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.171974 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.172127 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.172213 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.172359 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.172435 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.172588 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.172664 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.172820 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.172904 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.173075 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.173151 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.173354 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.173437 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.173574 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.173665 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.173812 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.173890 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.174047 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.174125 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.174287 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.174496 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.175093 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.175216 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.175292 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.175417 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.175534 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.175647 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.175766 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.175876 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.176000 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.176106 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.176232 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.176335 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.176468 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.176569 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.176698 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.176797 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.176939 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.177018 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.177175 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.177254 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.177450 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.177530 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.177692 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.177769 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.177950 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.178030 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.178212 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.178336 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.178454 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.178563 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.178688 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] auth.external = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.178783 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] auth.external = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.178921 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.179007 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.179218 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.179297 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.179450 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.179528 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.179681 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] auth.password = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.179761 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] auth.password = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.179911 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] auth.token = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.179989 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] auth.token = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.180151 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.180231 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.180382 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.180467 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.180625 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.180702 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.180851 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.180938 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.181098 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.181179 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.181360 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.181445 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.181592 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.181669 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.181823 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] credential.caching = True {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.181902 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] credential.caching = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.182066 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.182144 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.182313 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.182392 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.182559 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.182639 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.182800 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.182877 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.183040 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.183117 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.183270 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.183349 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.183511 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.183587 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.183760 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.183837 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.183992 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.184070 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.184241 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.184320 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.184521 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.184605 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.184764 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.184851 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.185026 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.185104 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.185265 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.185358 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.185541 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.185628 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.185772 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] federation.caching = True {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.185859 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] federation.caching = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.186008 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.186091 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.186247 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.186332 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.186490 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.186567 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.186716 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.186794 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.186960 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.187041 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.187195 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.187272 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.187444 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.187529 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.187680 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.187763 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.187929 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.188007 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.188165 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.188249 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.188406 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.188483 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.188638 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] identity.caching = True {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.188722 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] identity.caching = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.188882 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.189036 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.189112 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.189183 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.189371 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.189454 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.189605 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.189695 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.189849 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.189927 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.190097 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.190176 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.190335 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.190423 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.190579 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.190657 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.190828 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.190906 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.191058 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.191145 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.191287 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.191404 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.191513 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.191592 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.191755 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.191844 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.191995 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.192073 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.192238 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.192317 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.192489 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.192581 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.192736 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.192814 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.193002 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.193081 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.193235 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.193336 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.193493 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.193571 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.193726 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.193802 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.193964 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.194058 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.194198 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.194278 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.194434 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.194511 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.194675 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.194764 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.194916 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.194994 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.195162 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.195240 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.195389 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.195476 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.195634 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.195712 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.195875 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.195954 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.196106 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.196193 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.196349 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.196427 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.196591 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.196669 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.196820 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.196907 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.197052 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.197130 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.197278 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.197415 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.197535 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.197615 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.197769 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.197847 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.198008 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.198086 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.198242 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.198323 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.198475 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.198553 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.198719 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.198797 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.198963 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.199043 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.199190 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.199276 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.199418 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.199497 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.199661 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.199741 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.199905 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.199983 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.200138 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.200215 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.200368 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.200448 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.200600 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.200677 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.200828 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.user = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.200917 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ldap.user = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.201063 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.201142 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.201330 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.201412 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.201567 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.201656 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.201810 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.201892 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.202053 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.202131 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.202296 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.202374 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.202523 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.202602 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.202750 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.202827 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.202979 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.203057 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.203210 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.203290 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.203441 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.203519 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.203667 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.203744 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.203907 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.203986 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.204148 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.204226 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.204389 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.204466 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.204628 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.204708 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.204869 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.204947 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.205094 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.205172 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.205356 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.205438 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.205590 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.206162 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.206350 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.206429 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.206581 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.206662 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.206810 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.206888 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.207053 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.207130 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.207295 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.207374 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.207529 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.207608 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.207781 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.207859 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.208010 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.208088 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.208248 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.208324 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.208481 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.208559 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.208711 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.208791 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.208945 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.209023 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.209207 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.209285 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.209522 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.209604 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.209764 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.209852 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.209984 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.210062 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.210222 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] resource.caching = True {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.210302 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] resource.caching = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.210468 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.210546 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.210712 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.210790 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.210982 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.211061 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.211226 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.211303 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.211472 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.211550 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.211702 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.211828 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.211943 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.212021 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.212176 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.212254 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.212401 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.212476 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.212627 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] role.caching = True {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.212706 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] role.caching = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.212869 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] role.driver = sql {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.212957 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] role.driver = sql {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.213095 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.213174 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.213364 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.213445 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.213609 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.213700 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.213853 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.213931 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.214097 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.214175 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.214339 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.214426 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.214583 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.214661 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.214822 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.214899 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.215064 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.215151 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.215295 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.215373 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.215535 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.215612 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.215776 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.215863 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.216017 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.216096 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.216259 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.216337 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.216656 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.216741 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.216891 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.216968 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.217141 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.217218 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.217414 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.217506 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.217663 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.217741 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.217911 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.217990 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.218143 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.218230 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.218394 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.218472 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.218645 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.218722 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.218881 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.218961 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.219112 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.219189 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.219341 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.219418 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.219570 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.219649 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.219817 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.219894 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.220068 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.220147 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.220314 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.220394 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.220546 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.220628 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.220778 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.220856 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.221004 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.221085 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.221234 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] token.caching = True {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.221357 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] token.caching = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.222070 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.222215 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.222379 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.222461 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.222623 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.222701 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.222884 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.222962 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.223134 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.223214 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.223372 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.223450 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.223615 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.223692 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.224107 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.224189 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.224352 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.224430 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.224582 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.224660 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.224818 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.224897 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.225045 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.225122 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.225286 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.225425 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.225582 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.225663 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.225809 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.225896 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.226047 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.226135 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.226309 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.226388 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.226567 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] database.connection = **** {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.226654 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] database.connection = **** {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.226807 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.226886 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.227051 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.227129 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.227286 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.227378 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.227524 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.227602 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.227759 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.227836 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.227999 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.228088 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.228238 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.228316 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.228476 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.228554 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.228722 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.228808 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.228953 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.229031 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.229194 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.229271 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.229447 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.229539 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.229693 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.229773 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.229925 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.230003 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.230167 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.230254 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.230395 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.230472 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.230630 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.230707 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.230863 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.230949 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:16:47.231022 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 01 17:16:47.231093 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 01 17:16:47.260197 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81501) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:16:47.260404 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:16:47.260509 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81501) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:16:47.260588 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:16:47.260662 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81501) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:16:47.260736 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:16:47.269549 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:16:47.269745 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81501) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:16:47.298153 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:16:47.298285 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81501) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:16:47.298380 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:16:47.298467 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81501) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:16:47.353251 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:16:47.353448 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81501) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:16:47.403058 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81501) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:16:47.403283 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81501) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:16:47.404028 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:16:47.404262 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:16:47.559753 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:16:47.559986 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:16:47.560205 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:16:47.560427 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:16:47.560647 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:16:47.560865 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:16:47.563984 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81501) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:16:47.564214 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81501) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:16:47.564431 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81501) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:16:47.564645 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81501) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:16:47.564858 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81501) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:16:47.565610 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81501) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:16:47.585215 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:16:47.585346 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81501) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:16:47.585475 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:16:47.585560 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81501) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:16:47.585721 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:16:47.585807 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81501) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:16:47.585927 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81501) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:16:47.586012 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:16:47.586142 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:16:47.586233 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81501) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:16:47.586354 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:16:47.586438 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81501) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:16:47.586566 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81501) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:16:47.586647 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:16:47.598227 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81501) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:16:47.598471 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:16:47.611205 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81501) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:16:47.611435 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81501) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:16:47.611525 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:16:47.611647 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81501) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:16:47.611732 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:16:47.611865 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81501) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:16:47.611949 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:16:47.612624 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81501) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:16:47.612712 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:16:47.612838 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81501) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:16:47.612924 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:16:47.613046 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81501) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:16:47.613130 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:16:47.613267 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:16:47.629984 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.630190 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 01 17:16:47.630760 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.630962 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 01 17:16:47.631045 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.631186 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 01 17:16:47.631321 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.631530 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.631612 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.631731 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 01 17:16:47.631836 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.632010 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.632094 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.632277 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.632539 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.632796 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.641550 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.641831 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.642100 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.642379 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.642466 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.642702 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.642976 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.643284 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.649356 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81501) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:16:47.650310 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:16:47.652245 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81501) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:16:47.652528 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.652813 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.653098 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.653218 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:16:47.653403 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.653529 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.653667 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.653819 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.654050 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.654131 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.654330 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.654421 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.654604 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.654686 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.654880 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.654961 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.655148 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.655232 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.655422 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.655701 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.655972 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.679408 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.680249 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.680873 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81501) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:16:47.681154 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.681755 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:16:47.682045 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.685860 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81501) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:16:47.686145 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.686428 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.686692 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:16:47.686936 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.687221 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.693443 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81501) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:16:47.693728 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.694325 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:16:47.694615 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.697849 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81501) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:16:47.698140 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.698423 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.698705 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.698856 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:16:47.699047 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.699261 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.699458 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.699657 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.699854 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.700065 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.700273 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.700467 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.702051 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.702349 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.702639 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.718614 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.718910 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.719112 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.719408 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.723055 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81501) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:16:47.723370 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:16:47.726476 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.726770 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.726922 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.727075 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.730992 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81501) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:16:47.731457 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:16:47.734275 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81501) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:16:47.734643 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:16:47.737435 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.737737 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.737887 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.738024 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.738156 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.738286 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.738417 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.738558 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.738691 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.738821 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.738948 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.739360 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.755486 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81501) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:16:47.755768 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.755908 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:16:47.756048 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.756203 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.756332 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.756476 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.756608 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.763283 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.763447 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.764814 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.764995 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.766379 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.766580 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.766719 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.766846 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.770068 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81501) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:16:47.770250 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:16:47.770394 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.770530 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.770665 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.770810 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.770949 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.771083 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.771215 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.771356 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.782622 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81501) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:16:47.782811 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:16:47.782953 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.783086 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.783215 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.783356 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.783494 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.783626 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.783754 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.783882 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.784010 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.784145 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.784273 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.784399 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.799767 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81501) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:16:47.799920 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:16:47.802725 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81501) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:16:47.802897 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:16:47.805712 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.805864 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.807339 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.807521 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.808757 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81501) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:16:47.808913 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:16:47.809054 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.809193 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.809365 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.809579 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.816294 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.816435 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.817918 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81501) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:16:47.818010 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:16:47.820844 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.820941 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.821139 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.821238 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.821463 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.821547 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.821744 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.821825 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.829691 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81501) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:16:47.829788 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:16:47.829980 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.830069 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.830260 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.830342 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.837860 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:16:47.837962 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81501) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:16:47.838150 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.838233 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.838439 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.838535 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.838715 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.838796 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.838986 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.839068 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.839256 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.839355 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.839529 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.839621 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.839820 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.839908 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.840108 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.840264 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.840392 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.840549 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.840689 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.840827 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.840918 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.841115 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.841208 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.841407 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.841499 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.841684 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:16:47.875887 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x56517b8a99c0 pid: 81502 (default app) Sep 01 17:16:47.876147 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x56517b8a99c0 pid: 81501 (default app) Sep 01 17:16:47.880015 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e1d05a-f628-4385-bb28-648030bb3f3b None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:16:47.880350 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e1d05a-f628-4385-bb28-648030bb3f3b None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:16:47.880670 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e1d05a-f628-4385-bb28-648030bb3f3b None None] PATH_INFO: `/v3` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:16:47.884853 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1/1] 10.222.0.102 () {50 vars in 796 bytes} [Thu Sep 1 17:16:47 2022] GET /identity/v3/ => generated 256 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:16:54.453231 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b44ee4d-cdd6-4964-9a4c-53d727eb4600 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:16:54.453765 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b44ee4d-cdd6-4964-9a4c-53d727eb4600 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:16:54.453868 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b44ee4d-cdd6-4964-9a4c-53d727eb4600 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:16:54.458298 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1/2] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:16:54 2022] GET /identity => generated 271 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Sep 01 17:16:54.463803 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f00618-8a7d-4a5a-9296-ebe1381fbb17 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:16:54.464366 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f00618-8a7d-4a5a-9296-ebe1381fbb17 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:16:54.464366 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f00618-8a7d-4a5a-9296-ebe1381fbb17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:16:54.536058 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: INFO dbcounter [None req-98f00618-8a7d-4a5a-9296-ebe1381fbb17 None None] Registered counter for database keystone Sep 01 17:16:54.555743 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG oslo_db.sqlalchemy.engines [None req-98f00618-8a7d-4a5a-9296-ebe1381fbb17 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81502) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 01 17:16:54.561956 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writer thread running {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 01 17:16:54.740587 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG passlib.handlers.bcrypt [None req-98f00618-8a7d-4a5a-9296-ebe1381fbb17 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81502) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 01 17:16:54.740919 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG passlib.handlers.bcrypt [None req-98f00618-8a7d-4a5a-9296-ebe1381fbb17 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81502) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 01 17:16:54.798978 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-98f00618-8a7d-4a5a-9296-ebe1381fbb17 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:16:54.801095 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-98f00618-8a7d-4a5a-9296-ebe1381fbb17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:16:54.930262 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 2/3] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:16:54 2022] POST /identity/v3/auth/tokens => generated 915 bytes in 467 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Sep 01 17:16:55.377232 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b44ee4d-cdd6-4964-9a4c-53d727eb4600 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:16:55.446713 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: INFO dbcounter [None req-3b44ee4d-cdd6-4964-9a4c-53d727eb4600 None None] Registered counter for database keystone Sep 01 17:16:55.473704 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG oslo_db.sqlalchemy.engines [None req-3b44ee4d-cdd6-4964-9a4c-53d727eb4600 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81501) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 01 17:16:55.480113 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writer thread running {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 01 17:16:55.599191 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b44ee4d-cdd6-4964-9a4c-53d727eb4600 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:16:55.603758 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83db3366-6ea2-445b-90d7-1861efae8d2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:16:55.604380 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83db3366-6ea2-445b-90d7-1861efae8d2b admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:16:55.604662 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83db3366-6ea2-445b-90d7-1861efae8d2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:16:55.604938 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83db3366-6ea2-445b-90d7-1861efae8d2b admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:16:55.613044 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83db3366-6ea2-445b-90d7-1861efae8d2b admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:16:55.664270 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: WARNING keystone.common.rbac_enforcer.enforcer [None req-83db3366-6ea2-445b-90d7-1861efae8d2b admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 01 17:16:55.664554 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83db3366-6ea2-445b-90d7-1861efae8d2b admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:16:55.674059 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 2/4] 10.222.0.102 () {58 vars in 1329 bytes} [Thu Sep 1 17:16:55 2022] GET /identity/v3/auth/tokens => generated 915 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 01 17:16:55.680049 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f00618-8a7d-4a5a-9296-ebe1381fbb17 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:16:55.694328 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f00618-8a7d-4a5a-9296-ebe1381fbb17 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:16:55.698737 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1df300-418f-48c9-a89f-ebce40137177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:16:55.699825 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1df300-418f-48c9-a89f-ebce40137177 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:16:55.700111 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1df300-418f-48c9-a89f-ebce40137177 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:16:55.700387 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1df300-418f-48c9-a89f-ebce40137177 admin admin] PATH_INFO: `/v3/projects/ace9b3845813477eb192c158600d0beb` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:16:55.701991 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1df300-418f-48c9-a89f-ebce40137177 admin admin] RBAC: Authorizing `identity:get_project(project_id=ace9b3845813477eb192c158600d0beb)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:16:55.755496 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: WARNING keystone.common.rbac_enforcer.enforcer [None req-9d1df300-418f-48c9-a89f-ebce40137177 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 01 17:16:55.755850 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1df300-418f-48c9-a89f-ebce40137177 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:16:55.760445 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 3/5] 10.222.0.102 () {56 vars in 1212 bytes} [Thu Sep 1 17:16:55 2022] GET /identity/v3/projects/ace9b3845813477eb192c158600d0beb => generated 345 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:16:57.614732 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196dbe43-dde0-464c-9691-9e5cdc37d419 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:16:57.615345 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196dbe43-dde0-464c-9691-9e5cdc37d419 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:16:57.615345 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196dbe43-dde0-464c-9691-9e5cdc37d419 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:16:57.619394 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 3/6] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:16:57 2022] GET /identity => generated 271 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Sep 01 17:16:57.624739 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97d2654-b9b2-4c53-914a-6f5bdb932651 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:16:57.625115 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97d2654-b9b2-4c53-914a-6f5bdb932651 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:16:57.625293 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97d2654-b9b2-4c53-914a-6f5bdb932651 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:16:57.644955 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-b97d2654-b9b2-4c53-914a-6f5bdb932651 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:16:57.646506 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-b97d2654-b9b2-4c53-914a-6f5bdb932651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:16:57.654219 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 4/7] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:16:57 2022] POST /identity/v3/auth/tokens => generated 915 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 01 17:16:57.923942 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196dbe43-dde0-464c-9691-9e5cdc37d419 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:16:57.936166 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196dbe43-dde0-464c-9691-9e5cdc37d419 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:16:57.940386 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80d094f-a88e-4f02-b393-4d837908310c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:16:57.940972 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80d094f-a88e-4f02-b393-4d837908310c admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:16:57.941247 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80d094f-a88e-4f02-b393-4d837908310c admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:16:57.941545 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80d094f-a88e-4f02-b393-4d837908310c admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:16:57.949075 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80d094f-a88e-4f02-b393-4d837908310c admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:16:57.950873 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80d094f-a88e-4f02-b393-4d837908310c admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:16:57.959906 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 4/8] 10.222.0.102 () {58 vars in 1329 bytes} [Thu Sep 1 17:16:57 2022] GET /identity/v3/auth/tokens => generated 915 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 17:16:57.965278 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97d2654-b9b2-4c53-914a-6f5bdb932651 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:16:57.975101 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97d2654-b9b2-4c53-914a-6f5bdb932651 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:16:57.979383 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13532f3-1bed-4b8e-aeda-8a11a9dd943e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:16:57.979996 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13532f3-1bed-4b8e-aeda-8a11a9dd943e admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:16:57.980274 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13532f3-1bed-4b8e-aeda-8a11a9dd943e admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:16:57.980544 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13532f3-1bed-4b8e-aeda-8a11a9dd943e admin admin] PATH_INFO: `/v3/users/e6c9197e0f844432a03b9e75fc87712b` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:16:57.981869 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13532f3-1bed-4b8e-aeda-8a11a9dd943e admin admin] RBAC: Authorizing `identity:get_user(user_id=e6c9197e0f844432a03b9e75fc87712b)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:16:57.983669 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13532f3-1bed-4b8e-aeda-8a11a9dd943e admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:16:57.985177 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 5/9] 10.222.0.102 () {56 vars in 1203 bytes} [Thu Sep 1 17:16:57 2022] GET /identity/v3/users/e6c9197e0f844432a03b9e75fc87712b => generated 248 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:16:59.941264 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf785073-0687-40cb-a324-dd2f4d121856 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:16:59.941809 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf785073-0687-40cb-a324-dd2f4d121856 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:16:59.941896 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf785073-0687-40cb-a324-dd2f4d121856 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:16:59.943211 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 5/10] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:16:59 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:16:59.948448 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f08708-7864-4676-b24c-e870251e5edd None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:16:59.948900 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f08708-7864-4676-b24c-e870251e5edd None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:16:59.949000 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f08708-7864-4676-b24c-e870251e5edd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:16:59.967881 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-71f08708-7864-4676-b24c-e870251e5edd None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:16:59.969066 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-71f08708-7864-4676-b24c-e870251e5edd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:16:59.987341 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1341d3-cc6f-4676-8e54-8f56dd0da948 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:16:59.987624 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1341d3-cc6f-4676-8e54-8f56dd0da948 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:16:59.987898 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1341d3-cc6f-4676-8e54-8f56dd0da948 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:16:59.989207 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 6/11] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:16:59 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Sep 01 17:16:59.994468 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a97539-51d0-415b-a33a-1a2de654e8ef None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:16:59.994749 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a97539-51d0-415b-a33a-1a2de654e8ef None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:16:59.995018 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a97539-51d0-415b-a33a-1a2de654e8ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:00.036451 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG passlib.handlers.bcrypt [None req-92a97539-51d0-415b-a33a-1a2de654e8ef None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81501) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 01 17:17:00.036840 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG passlib.handlers.bcrypt [None req-92a97539-51d0-415b-a33a-1a2de654e8ef None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81501) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 01 17:17:00.051830 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 6/12] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:16:59 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:17:00.086232 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-92a97539-51d0-415b-a33a-1a2de654e8ef None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:17:00.087811 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-92a97539-51d0-415b-a33a-1a2de654e8ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:17:00.187663 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 7/13] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:16:59 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 17:17:00.316935 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f08708-7864-4676-b24c-e870251e5edd None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:00.340955 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:00.424363 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f08708-7864-4676-b24c-e870251e5edd None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:00.458452 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a97539-51d0-415b-a33a-1a2de654e8ef None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:00.458681 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f8cec9-ca41-4396-8c42-5fb4d1c2537f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:00.459335 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f8cec9-ca41-4396-8c42-5fb4d1c2537f None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:00.459620 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f8cec9-ca41-4396-8c42-5fb4d1c2537f None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:00.459898 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f8cec9-ca41-4396-8c42-5fb4d1c2537f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:00.471487 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f8cec9-ca41-4396-8c42-5fb4d1c2537f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:17:00.480666 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-b2f8cec9-ca41-4396-8c42-5fb4d1c2537f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:17:00.480666 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:17:00.480666 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:17:00.480666 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:17:00.480666 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:17:00.480666 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:17:00.480666 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:17:00.480666 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:17:00.480666 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:17:00.480666 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:17:00.480666 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:17:00.480666 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:17:00.480666 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:17:00.480666 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:17:00.480666 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:17:00.480666 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:17:00.481630 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:17:00.481630 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:17:00.481630 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:17:00.481630 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:17:00.481630 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:17:00.481630 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:17:00.481630 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:17:00.481630 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:17:00.481630 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:17:00.481630 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:17:00.481630 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:17:00.481630 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:17:00.481630 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:17:00.481630 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:17:00.481630 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:17:00.481630 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:17:00.482455 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:17:00.482455 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:17:00.482455 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:17:00.482455 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:17:00.482455 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:17:00.482455 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:17:00.482455 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:17:00.482455 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:17:00.482455 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:17:00.482455 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:17:00.482455 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:17:00.482455 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:17:00.482455 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 7/14] 10.222.0.102 () {56 vars in 1101 bytes} [Thu Sep 1 17:17:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:17:00.487468 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f8cec9-ca41-4396-8c42-5fb4d1c2537f None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:00.562515 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a97539-51d0-415b-a33a-1a2de654e8ef None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:00.565118 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:00.568838 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:00.588796 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f8cec9-ca41-4396-8c42-5fb4d1c2537f None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:00.602653 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e57a1ff-b589-46a3-b597-afb93c78f0ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:00.603443 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e57a1ff-b589-46a3-b597-afb93c78f0ae None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:00.603718 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e57a1ff-b589-46a3-b597-afb93c78f0ae None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:00.603987 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e57a1ff-b589-46a3-b597-afb93c78f0ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:00.623625 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88fb037-2f43-4b62-a68a-808046d90cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:00.624320 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88fb037-2f43-4b62-a68a-808046d90cba None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:00.624604 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88fb037-2f43-4b62-a68a-808046d90cba None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:00.624877 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88fb037-2f43-4b62-a68a-808046d90cba None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:00.625583 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88fb037-2f43-4b62-a68a-808046d90cba None admin] RBAC: Adding query filter params (name=admin) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:17:00.625893 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88fb037-2f43-4b62-a68a-808046d90cba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:00.627287 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88fb037-2f43-4b62-a68a-808046d90cba None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:00.638961 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 8/15] 10.222.0.102 () {56 vars in 1104 bytes} [Thu Sep 1 17:17:00 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:17:00.644579 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88fb037-2f43-4b62-a68a-808046d90cba None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:00.671893 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e57a1ff-b589-46a3-b597-afb93c78f0ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:00.673528 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e57a1ff-b589-46a3-b597-afb93c78f0ae None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:00.742152 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88fb037-2f43-4b62-a68a-808046d90cba None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:00.771056 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 8/16] 10.222.0.102 () {58 vars in 1287 bytes} [Thu Sep 1 17:17:00 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 01 17:17:00.776120 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9d48ee-66b8-460e-a79b-534ff7cade44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:00.776609 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e57a1ff-b589-46a3-b597-afb93c78f0ae None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:00.776760 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9d48ee-66b8-460e-a79b-534ff7cade44 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:00.777041 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9d48ee-66b8-460e-a79b-534ff7cade44 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:00.777325 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9d48ee-66b8-460e-a79b-534ff7cade44 None admin] PATH_INFO: `/v3/users/e6c9197e0f844432a03b9e75fc87712b` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:00.778482 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9d48ee-66b8-460e-a79b-534ff7cade44 None admin] RBAC: Authorizing `identity:get_user(user_id=e6c9197e0f844432a03b9e75fc87712b)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:00.779345 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:00.779939 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9d48ee-66b8-460e-a79b-534ff7cade44 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:00.781742 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 9/17] 10.222.0.102 () {56 vars in 1182 bytes} [Thu Sep 1 17:17:00 2022] GET /identity/v3/users/e6c9197e0f844432a03b9e75fc87712b => generated 248 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:17:00.787782 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9d48ee-66b8-460e-a79b-534ff7cade44 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:00.802284 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:00.877763 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e57a1ff-b589-46a3-b597-afb93c78f0ae None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:00.895004 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9d48ee-66b8-460e-a79b-534ff7cade44 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:00.911820 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c174a191-14af-4ef7-979c-9fa16d7db5f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:00.912673 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c174a191-14af-4ef7-979c-9fa16d7db5f8 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:00.912956 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c174a191-14af-4ef7-979c-9fa16d7db5f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:00.913418 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c174a191-14af-4ef7-979c-9fa16d7db5f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:00.930808 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa143c2d-f6b6-4220-9a57-cbf8aeab6ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:00.931435 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa143c2d-f6b6-4220-9a57-cbf8aeab6ff9 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:00.931721 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa143c2d-f6b6-4220-9a57-cbf8aeab6ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:00.931859 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c174a191-14af-4ef7-979c-9fa16d7db5f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:00.931991 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa143c2d-f6b6-4220-9a57-cbf8aeab6ff9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:00.933107 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa143c2d-f6b6-4220-9a57-cbf8aeab6ff9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:00.933251 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c174a191-14af-4ef7-979c-9fa16d7db5f8 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:00.934572 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa143c2d-f6b6-4220-9a57-cbf8aeab6ff9 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:00.936116 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 10/18] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:17:00 2022] GET /identity/v3/domains/default => generated 200 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:17:00.941953 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa143c2d-f6b6-4220-9a57-cbf8aeab6ff9 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:00.951385 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-c174a191-14af-4ef7-979c-9fa16d7db5f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:17:00.951385 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:17:00.951385 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:17:00.951385 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:17:00.951385 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:17:00.951385 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:17:00.951385 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:17:00.951385 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:17:00.951385 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:17:00.951385 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:17:00.951385 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:17:00.951385 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:17:00.951385 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:17:00.951385 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:17:00.951385 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:17:00.951385 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:17:00.952725 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:17:00.952725 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:17:00.952725 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:17:00.952725 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:17:00.952725 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:17:00.952725 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:17:00.952725 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:17:00.952725 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:17:00.952725 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:17:00.952725 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:17:00.952725 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:17:00.952725 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:17:00.952725 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:17:00.952725 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:17:00.952725 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:17:00.952725 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:17:00.952725 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:17:00.953730 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:17:00.953730 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:17:00.953730 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:17:00.953730 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 9/19] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:17:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:17:00.958737 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c174a191-14af-4ef7-979c-9fa16d7db5f8 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:01.014947 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:01.027336 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:01.046992 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa143c2d-f6b6-4220-9a57-cbf8aeab6ff9 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:01.062195 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c174a191-14af-4ef7-979c-9fa16d7db5f8 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:01.081320 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebf1885-43bf-4905-9b5e-8f38c5530e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:01.082581 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebf1885-43bf-4905-9b5e-8f38c5530e99 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:01.082581 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebf1885-43bf-4905-9b5e-8f38c5530e99 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:01.082870 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebf1885-43bf-4905-9b5e-8f38c5530e99 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:01.083600 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebf1885-43bf-4905-9b5e-8f38c5530e99 None admin] RBAC: Adding query filter params (role.id=36577a6f46d642d4b81805ede9872490, scope.domain.id=default, user.id=e6c9197e0f844432a03b9e75fc87712b) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:17:01.083907 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebf1885-43bf-4905-9b5e-8f38c5530e99 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:01.085381 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebf1885-43bf-4905-9b5e-8f38c5530e99 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:01.092025 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 11/20] 10.222.0.102 () {56 vars in 1327 bytes} [Thu Sep 1 17:17:00 2022] GET /identity/v3/role_assignments?user.id=e6c9197e0f844432a03b9e75fc87712b&scope.domain.id=default&role.id=36577a6f46d642d4b81805ede9872490 => generated 234 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:17:01.097931 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f01d5a3-d5db-40c3-a163-eee705bec08f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:01.098557 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f01d5a3-d5db-40c3-a163-eee705bec08f None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:01.098836 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f01d5a3-d5db-40c3-a163-eee705bec08f None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:01.099109 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f01d5a3-d5db-40c3-a163-eee705bec08f None admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:01.099781 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f01d5a3-d5db-40c3-a163-eee705bec08f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:17:01.100348 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f01d5a3-d5db-40c3-a163-eee705bec08f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:01.101966 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f01d5a3-d5db-40c3-a163-eee705bec08f None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:01.121956 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 10/21] 10.222.0.102 () {56 vars in 1114 bytes} [Thu Sep 1 17:17:00 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:17:03.280658 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96226b7e-9044-47d5-81a4-d3fc50e98fb3 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:03.281161 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96226b7e-9044-47d5-81a4-d3fc50e98fb3 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:03.281161 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96226b7e-9044-47d5-81a4-d3fc50e98fb3 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:03.282502 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 12/22] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:17:03 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:17:03.294783 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b72fdf2-7317-49b9-9105-29d21923e6d2 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:03.295235 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b72fdf2-7317-49b9-9105-29d21923e6d2 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:03.295397 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b72fdf2-7317-49b9-9105-29d21923e6d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:03.321177 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee1c519-2c27-4b7d-baab-a4a395e5d468 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:03.321493 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee1c519-2c27-4b7d-baab-a4a395e5d468 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:03.321755 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee1c519-2c27-4b7d-baab-a4a395e5d468 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:03.323293 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 13/23] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:17:03 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Sep 01 17:17:03.327965 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-0b72fdf2-7317-49b9-9105-29d21923e6d2 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:17:03.328401 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af50ec7-1d98-404f-9bd0-559edc1073a8 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:03.328693 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af50ec7-1d98-404f-9bd0-559edc1073a8 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:03.328973 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af50ec7-1d98-404f-9bd0-559edc1073a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:03.329208 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-0b72fdf2-7317-49b9-9105-29d21923e6d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:17:03.358998 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-6af50ec7-1d98-404f-9bd0-559edc1073a8 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:17:03.361552 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-6af50ec7-1d98-404f-9bd0-559edc1073a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:17:03.484020 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 11/24] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:17:03 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 17:17:03.504926 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 14/25] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:17:03 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 17:17:03.814243 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e11a281-197d-4beb-b5e1-243e3ac38a78 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:03.814773 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e11a281-197d-4beb-b5e1-243e3ac38a78 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:03.815265 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e11a281-197d-4beb-b5e1-243e3ac38a78 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:03.817121 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 12/26] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:17:03 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:17:03.822482 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a73f1f-a94f-4b0f-9bca-6f62280a196c None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:03.822776 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a73f1f-a94f-4b0f-9bca-6f62280a196c None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:03.823060 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a73f1f-a94f-4b0f-9bca-6f62280a196c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:03.834565 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f071dd8-f794-4839-ae07-5b06fae29f60 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:03.834849 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f071dd8-f794-4839-ae07-5b06fae29f60 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:03.835127 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f071dd8-f794-4839-ae07-5b06fae29f60 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:03.836469 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 13/27] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:17:03 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Sep 01 17:17:03.844361 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db720d06-5425-46c0-a032-6b825ae72e5e None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:03.844712 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db720d06-5425-46c0-a032-6b825ae72e5e None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:03.844995 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db720d06-5425-46c0-a032-6b825ae72e5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:03.846935 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-25a73f1f-a94f-4b0f-9bca-6f62280a196c None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:17:03.850990 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-25a73f1f-a94f-4b0f-9bca-6f62280a196c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:17:03.876834 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-db720d06-5425-46c0-a032-6b825ae72e5e None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:17:03.878085 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-db720d06-5425-46c0-a032-6b825ae72e5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:17:03.964825 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 15/28] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:17:03 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 17:17:03.971161 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a73f1f-a94f-4b0f-9bca-6f62280a196c None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:03.975559 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:03.983388 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:04.046842 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 14/29] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:17:03 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 17:17:04.047505 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db720d06-5425-46c0-a032-6b825ae72e5e None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:04.104444 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a73f1f-a94f-4b0f-9bca-6f62280a196c None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:04.168222 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b1d4de-8396-4461-9e7d-e5940d189938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:04.168876 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b1d4de-8396-4461-9e7d-e5940d189938 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:04.169161 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b1d4de-8396-4461-9e7d-e5940d189938 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:04.169476 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b1d4de-8396-4461-9e7d-e5940d189938 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:04.190065 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b1d4de-8396-4461-9e7d-e5940d189938 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:17:04.195644 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-d2b1d4de-8396-4461-9e7d-e5940d189938 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:17:04.195644 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:17:04.195644 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:17:04.195644 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:17:04.195644 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:17:04.195644 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:17:04.195644 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:17:04.195644 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:17:04.195644 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:17:04.195644 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:17:04.195644 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:17:04.195644 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:17:04.195644 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:17:04.195644 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:17:04.195644 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:17:04.195644 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:17:04.196532 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:17:04.196532 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:17:04.196532 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:17:04.196532 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:17:04.196532 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:17:04.196532 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:17:04.196532 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:17:04.196532 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:17:04.196532 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:17:04.196532 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:17:04.196532 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:17:04.196532 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:17:04.196532 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:17:04.196532 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:17:04.196532 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:17:04.196532 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:17:04.197286 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:17:04.197286 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:17:04.197286 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:17:04.197286 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:17:04.197286 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:17:04.197286 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:17:04.197286 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:17:04.197286 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:17:04.197286 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:17:04.197286 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:17:04.197286 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:17:04.197286 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:17:04.197286 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 16/30] 10.222.0.102 () {56 vars in 1101 bytes} [Thu Sep 1 17:17:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:17:04.204674 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b1d4de-8396-4461-9e7d-e5940d189938 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:04.285888 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:04.329284 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db720d06-5425-46c0-a032-6b825ae72e5e None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:04.352110 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b1d4de-8396-4461-9e7d-e5940d189938 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:04.387115 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4cfe54-1378-4518-85ec-19d7b382d246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:04.387677 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4cfe54-1378-4518-85ec-19d7b382d246 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:04.387958 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4cfe54-1378-4518-85ec-19d7b382d246 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:04.388235 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4cfe54-1378-4518-85ec-19d7b382d246 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:04.399750 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4cfe54-1378-4518-85ec-19d7b382d246 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:04.401229 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4cfe54-1378-4518-85ec-19d7b382d246 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:04.422161 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-6a4cfe54-1378-4518-85ec-19d7b382d246 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:17:04.422161 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:17:04.422161 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:17:04.422161 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:17:04.422161 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:17:04.422161 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:17:04.422161 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:17:04.422161 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:17:04.422161 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:17:04.422161 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:17:04.422161 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:17:04.422161 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:17:04.422161 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:17:04.422161 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:17:04.422161 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:17:04.422161 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:17:04.423075 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:17:04.423075 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:17:04.423075 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:17:04.423075 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:17:04.423075 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:17:04.423075 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:17:04.423075 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:17:04.423075 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:17:04.423075 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:17:04.423075 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:17:04.423075 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:17:04.423075 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:17:04.423075 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:17:04.423075 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:17:04.423075 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:17:04.423075 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:17:04.423075 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:17:04.423848 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:17:04.423848 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:17:04.423848 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:17:04.427838 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 15/31] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:17:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 380 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:17:04.429233 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4cfe54-1378-4518-85ec-19d7b382d246 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:04.434775 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13495566-6176-42e9-9c23-16978a518801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:04.435407 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13495566-6176-42e9-9c23-16978a518801 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:04.435687 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13495566-6176-42e9-9c23-16978a518801 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:04.435962 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13495566-6176-42e9-9c23-16978a518801 None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:04.436637 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13495566-6176-42e9-9c23-16978a518801 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:17:04.436942 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13495566-6176-42e9-9c23-16978a518801 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:04.439973 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13495566-6176-42e9-9c23-16978a518801 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:04.448798 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 17/32] 10.222.0.102 () {56 vars in 1104 bytes} [Thu Sep 1 17:17:04 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:17:04.449895 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13495566-6176-42e9-9c23-16978a518801 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:04.469203 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:04.559287 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4cfe54-1378-4518-85ec-19d7b382d246 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:04.594347 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13495566-6176-42e9-9c23-16978a518801 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:04.596692 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48917e4-9eed-44e3-8383-059fc1749d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:04.597524 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48917e4-9eed-44e3-8383-059fc1749d0f None admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:04.597976 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48917e4-9eed-44e3-8383-059fc1749d0f None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:04.598386 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48917e4-9eed-44e3-8383-059fc1749d0f None admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:04.599201 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48917e4-9eed-44e3-8383-059fc1749d0f None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:04.600974 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48917e4-9eed-44e3-8383-059fc1749d0f None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:04.625067 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:04.636927 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7c7bd9-2322-49e8-9738-0b9ea99fa44b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:04.637690 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7c7bd9-2322-49e8-9738-0b9ea99fa44b None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:04.637915 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7c7bd9-2322-49e8-9738-0b9ea99fa44b None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:04.638189 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7c7bd9-2322-49e8-9738-0b9ea99fa44b None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:04.638892 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7c7bd9-2322-49e8-9738-0b9ea99fa44b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:17:04.639202 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7c7bd9-2322-49e8-9738-0b9ea99fa44b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:04.640840 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7c7bd9-2322-49e8-9738-0b9ea99fa44b None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:04.662958 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 18/33] 10.222.0.102 () {56 vars in 1114 bytes} [Thu Sep 1 17:17:04 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:04.664051 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7c7bd9-2322-49e8-9738-0b9ea99fa44b None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:04.743483 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 16/34] 10.222.0.102 () {60 vars in 1136 bytes} [Thu Sep 1 17:17:04 2022] POST /identity/v3/roles => generated 226 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:17:04.744608 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48917e4-9eed-44e3-8383-059fc1749d0f None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:04.805334 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7c7bd9-2322-49e8-9738-0b9ea99fa44b None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:04.896885 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc161a8-3d58-42c6-88e5-398de6a82763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:04.897552 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc161a8-3d58-42c6-88e5-398de6a82763 None admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:04.897834 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc161a8-3d58-42c6-88e5-398de6a82763 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:04.898103 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc161a8-3d58-42c6-88e5-398de6a82763 None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:04.898799 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc161a8-3d58-42c6-88e5-398de6a82763 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:04.900209 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc161a8-3d58-42c6-88e5-398de6a82763 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:04.920644 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:04.940780 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48917e4-9eed-44e3-8383-059fc1749d0f None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:04.989149 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc51dfe-6b72-4588-b9f6-88b1e295a9e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:04.990181 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc51dfe-6b72-4588-b9f6-88b1e295a9e1 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:04.990664 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc51dfe-6b72-4588-b9f6-88b1e295a9e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:04.991136 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc51dfe-6b72-4588-b9f6-88b1e295a9e1 None admin] PATH_INFO: `/v3/users/e6c9197e0f844432a03b9e75fc87712b` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:04.993026 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc51dfe-6b72-4588-b9f6-88b1e295a9e1 None admin] RBAC: Authorizing `identity:get_user(user_id=e6c9197e0f844432a03b9e75fc87712b)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:04.995352 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc51dfe-6b72-4588-b9f6-88b1e295a9e1 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:04.997133 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 17/35] 10.222.0.102 () {56 vars in 1182 bytes} [Thu Sep 1 17:17:04 2022] GET /identity/v3/users/e6c9197e0f844432a03b9e75fc87712b => generated 248 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:04.997883 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc51dfe-6b72-4588-b9f6-88b1e295a9e1 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:05.036489 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 19/36] 10.222.0.102 () {60 vars in 1136 bytes} [Thu Sep 1 17:17:04 2022] POST /identity/v3/roles => generated 220 bytes in 373 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:17:05.038094 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bf5ad5-89d7-464d-9f45-7116b6f3700b None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:05.038301 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bf5ad5-89d7-464d-9f45-7116b6f3700b None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:05.038581 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bf5ad5-89d7-464d-9f45-7116b6f3700b None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:05.040014 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 20/37] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:17:05 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:17:05.040544 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bf5ad5-89d7-464d-9f45-7116b6f3700b None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:05.104020 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:05.143639 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc51dfe-6b72-4588-b9f6-88b1e295a9e1 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:05.184633 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bf5ad5-89d7-464d-9f45-7116b6f3700b None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:05.187346 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01a0f02-3ce7-4968-b084-826411172957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:05.187979 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01a0f02-3ce7-4968-b084-826411172957 None admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:05.188265 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01a0f02-3ce7-4968-b084-826411172957 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:05.188545 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01a0f02-3ce7-4968-b084-826411172957 None admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:05.189214 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01a0f02-3ce7-4968-b084-826411172957 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:05.191765 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01a0f02-3ce7-4968-b084-826411172957 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:05.226584 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 18/38] 10.222.0.102 () {60 vars in 1146 bytes} [Thu Sep 1 17:17:04 2022] POST /identity/v3/projects => generated 302 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:17:05.227730 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65be615-2eb7-4fbe-b189-143b24a77d17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:05.227944 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf64db8-0305-4a39-9b64-9c678436a617 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:05.228228 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf64db8-0305-4a39-9b64-9c678436a617 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:05.228380 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65be615-2eb7-4fbe-b189-143b24a77d17 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:05.228526 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf64db8-0305-4a39-9b64-9c678436a617 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:05.228683 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65be615-2eb7-4fbe-b189-143b24a77d17 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:05.228934 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65be615-2eb7-4fbe-b189-143b24a77d17 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:05.230241 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65be615-2eb7-4fbe-b189-143b24a77d17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:05.231851 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65be615-2eb7-4fbe-b189-143b24a77d17 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:05.233761 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 21/39] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:17:05 2022] GET /identity/v3/domains/default => generated 200 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:05.237407 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6ff4ee-1b92-4e4a-b859-af62a338e1fd None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:05.237407 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6ff4ee-1b92-4e4a-b859-af62a338e1fd None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:05.237407 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6ff4ee-1b92-4e4a-b859-af62a338e1fd None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:05.237407 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 22/40] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:17:05 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:17:05.242389 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6ff4ee-1b92-4e4a-b859-af62a338e1fd None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:05.257676 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-4cf64db8-0305-4a39-9b64-9c678436a617 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:17:05.259256 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-4cf64db8-0305-4a39-9b64-9c678436a617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:17:05.261760 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:05.353494 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 19/41] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:17:05 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:17:05.354966 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14fa1c3-81b8-4227-b0bf-97b64c40bf56 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:05.355251 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14fa1c3-81b8-4227-b0bf-97b64c40bf56 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:05.355607 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14fa1c3-81b8-4227-b0bf-97b64c40bf56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:05.384126 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-a14fa1c3-81b8-4227-b0bf-97b64c40bf56 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:17:05.385340 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-a14fa1c3-81b8-4227-b0bf-97b64c40bf56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:17:05.386363 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6ff4ee-1b92-4e4a-b859-af62a338e1fd None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:05.404263 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:05.438831 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973af425-8968-4fd3-932f-e5e4049e8446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:05.440212 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973af425-8968-4fd3-932f-e5e4049e8446 None admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:05.441077 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973af425-8968-4fd3-932f-e5e4049e8446 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:05.442032 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973af425-8968-4fd3-932f-e5e4049e8446 None admin] PATH_INFO: `/v3/domains/default/users/e6c9197e0f844432a03b9e75fc87712b/roles/36577a6f46d642d4b81805ede9872490` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:05.444633 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973af425-8968-4fd3-932f-e5e4049e8446 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=e6c9197e0f844432a03b9e75fc87712b, role_id=36577a6f46d642d4b81805ede9872490)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:05.447143 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973af425-8968-4fd3-932f-e5e4049e8446 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:05.466841 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 23/42] 10.222.0.102 () {58 vars in 1366 bytes} [Thu Sep 1 17:17:05 2022] PUT /identity/v3/domains/default/users/e6c9197e0f844432a03b9e75fc87712b/roles/36577a6f46d642d4b81805ede9872490 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:17:05.473369 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 20/43] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:17:05 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:17:05.623346 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973af425-8968-4fd3-932f-e5e4049e8446 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:05.746586 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14fa1c3-81b8-4227-b0bf-97b64c40bf56 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:05.818047 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:05.846218 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973af425-8968-4fd3-932f-e5e4049e8446 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:05.857370 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14fa1c3-81b8-4227-b0bf-97b64c40bf56 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:05.886264 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ea6480-82eb-4f71-870d-180831db29df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:05.886884 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ea6480-82eb-4f71-870d-180831db29df None admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:05.887169 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ea6480-82eb-4f71-870d-180831db29df None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:05.887501 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ea6480-82eb-4f71-870d-180831db29df None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:05.888178 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ea6480-82eb-4f71-870d-180831db29df None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:05.889586 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ea6480-82eb-4f71-870d-180831db29df None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:05.894419 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8448942a-3343-4bd5-a7d6-754bddc67a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:05.895248 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8448942a-3343-4bd5-a7d6-754bddc67a02 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:05.895687 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8448942a-3343-4bd5-a7d6-754bddc67a02 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:05.896079 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8448942a-3343-4bd5-a7d6-754bddc67a02 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:05.897125 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8448942a-3343-4bd5-a7d6-754bddc67a02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:05.898793 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8448942a-3343-4bd5-a7d6-754bddc67a02 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:05.900435 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 21/44] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:17:05 2022] GET /identity/v3/domains/default => generated 200 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:17:05.905338 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 24/45] 10.222.0.102 () {60 vars in 1136 bytes} [Thu Sep 1 17:17:05 2022] POST /identity/v3/roles => generated 224 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:17:05.905971 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8448942a-3343-4bd5-a7d6-754bddc67a02 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:06.020432 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8448942a-3343-4bd5-a7d6-754bddc67a02 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:06.062962 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5b6f6b-d7cd-4d3c-baf6-39ba75a7c223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:06.063582 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5b6f6b-d7cd-4d3c-baf6-39ba75a7c223 None admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:06.063867 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5b6f6b-d7cd-4d3c-baf6-39ba75a7c223 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:06.064144 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5b6f6b-d7cd-4d3c-baf6-39ba75a7c223 None admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:06.064800 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5b6f6b-d7cd-4d3c-baf6-39ba75a7c223 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:06.066329 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5b6f6b-d7cd-4d3c-baf6-39ba75a7c223 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:06.072357 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:06.096498 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 22/46] 10.222.0.102 () {60 vars in 1146 bytes} [Thu Sep 1 17:17:05 2022] POST /identity/v3/projects => generated 313 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:17:07.391562 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31761b63-c366-461b-a3ff-37540911ee01 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:07.392077 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31761b63-c366-461b-a3ff-37540911ee01 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:07.392157 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31761b63-c366-461b-a3ff-37540911ee01 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:07.393500 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 25/47] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:17:07 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:17:07.398728 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcbe717-edda-4cad-a87f-dda6b697cbc8 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:07.399124 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcbe717-edda-4cad-a87f-dda6b697cbc8 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:07.399391 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcbe717-edda-4cad-a87f-dda6b697cbc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:07.417739 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-3dcbe717-edda-4cad-a87f-dda6b697cbc8 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:17:07.418914 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-3dcbe717-edda-4cad-a87f-dda6b697cbc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:17:07.484533 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 23/48] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:17:07 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:17:07.747113 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31761b63-c366-461b-a3ff-37540911ee01 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:07.782716 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:07.852463 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31761b63-c366-461b-a3ff-37540911ee01 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:07.886725 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e55fbde-d1e8-4a1a-92a6-deae43d759f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:07.887360 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e55fbde-d1e8-4a1a-92a6-deae43d759f6 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:07.887638 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e55fbde-d1e8-4a1a-92a6-deae43d759f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:07.887912 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e55fbde-d1e8-4a1a-92a6-deae43d759f6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:07.897972 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e55fbde-d1e8-4a1a-92a6-deae43d759f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:17:07.902756 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-4e55fbde-d1e8-4a1a-92a6-deae43d759f6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:17:07.902756 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:17:07.902756 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:17:07.902756 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:17:07.902756 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:17:07.902756 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:17:07.902756 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:17:07.902756 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:17:07.902756 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:17:07.902756 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:17:07.902756 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:17:07.902756 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:17:07.902756 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:17:07.902756 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:17:07.902756 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:17:07.902756 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:17:07.903751 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:17:07.903751 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:17:07.903751 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:17:07.903751 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:17:07.903751 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:17:07.903751 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:17:07.903751 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:17:07.903751 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:17:07.903751 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:17:07.903751 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:17:07.903751 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:17:07.903751 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:17:07.903751 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:17:07.903751 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:17:07.903751 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:17:07.903751 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:17:07.904506 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:17:07.904506 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:17:07.904506 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:17:07.904506 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:17:07.904506 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:17:07.904506 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:17:07.904506 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:17:07.904506 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:17:07.904506 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:17:07.904506 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:17:07.904506 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:17:07.904506 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:17:07.904506 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 26/49] 10.222.0.102 () {56 vars in 1101 bytes} [Thu Sep 1 17:17:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:17:07.909577 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcbe717-edda-4cad-a87f-dda6b697cbc8 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:08.009448 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcbe717-edda-4cad-a87f-dda6b697cbc8 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:08.028816 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a92ddce-b378-4e86-ab71-2bab0164c2c4 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:08.029112 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a92ddce-b378-4e86-ab71-2bab0164c2c4 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:08.029426 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a92ddce-b378-4e86-ab71-2bab0164c2c4 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:08.030825 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 27/50] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:17:08 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Sep 01 17:17:08.036735 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50a7cf5-5716-467c-bcb7-78167d0329d7 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:08.037013 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50a7cf5-5716-467c-bcb7-78167d0329d7 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:08.037287 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50a7cf5-5716-467c-bcb7-78167d0329d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:08.056742 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-e50a7cf5-5716-467c-bcb7-78167d0329d7 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:17:08.058361 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-e50a7cf5-5716-467c-bcb7-78167d0329d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:17:08.129889 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 28/51] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:17:08 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 17:17:08.130611 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:08.148811 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b83dc9-5ad7-4987-a74d-66b1527de600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:08.149448 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b83dc9-5ad7-4987-a74d-66b1527de600 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:08.149728 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b83dc9-5ad7-4987-a74d-66b1527de600 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:08.150001 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b83dc9-5ad7-4987-a74d-66b1527de600 None admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:08.150660 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b83dc9-5ad7-4987-a74d-66b1527de600 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:17:08.150962 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b83dc9-5ad7-4987-a74d-66b1527de600 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:08.152325 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b83dc9-5ad7-4987-a74d-66b1527de600 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:08.163853 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 24/52] 10.222.0.102 () {56 vars in 1104 bytes} [Thu Sep 1 17:17:07 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:08.170193 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50a7cf5-5716-467c-bcb7-78167d0329d7 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:08.197612 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:08.278676 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50a7cf5-5716-467c-bcb7-78167d0329d7 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:08.311280 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb672e2c-50dc-4fa9-a6b1-c37bd3b3a3f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:08.311907 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb672e2c-50dc-4fa9-a6b1-c37bd3b3a3f8 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:08.312189 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb672e2c-50dc-4fa9-a6b1-c37bd3b3a3f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:08.312464 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb672e2c-50dc-4fa9-a6b1-c37bd3b3a3f8 None admin] PATH_INFO: `/v3/users/e6c9197e0f844432a03b9e75fc87712b` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:08.313613 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb672e2c-50dc-4fa9-a6b1-c37bd3b3a3f8 None admin] RBAC: Authorizing `identity:get_user(user_id=e6c9197e0f844432a03b9e75fc87712b)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:08.315054 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb672e2c-50dc-4fa9-a6b1-c37bd3b3a3f8 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:08.317188 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 29/53] 10.222.0.102 () {56 vars in 1182 bytes} [Thu Sep 1 17:17:08 2022] GET /identity/v3/users/e6c9197e0f844432a03b9e75fc87712b => generated 248 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:08.322979 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b83dc9-5ad7-4987-a74d-66b1527de600 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:08.403107 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb672e2c-50dc-4fa9-a6b1-c37bd3b3a3f8 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:08.422717 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b83dc9-5ad7-4987-a74d-66b1527de600 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:08.457656 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6422b60b-cd42-4a7b-b94f-16e55920ea22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:08.458287 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6422b60b-cd42-4a7b-b94f-16e55920ea22 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:08.458575 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6422b60b-cd42-4a7b-b94f-16e55920ea22 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:08.458851 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6422b60b-cd42-4a7b-b94f-16e55920ea22 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:08.459970 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6422b60b-cd42-4a7b-b94f-16e55920ea22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:08.461492 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6422b60b-cd42-4a7b-b94f-16e55920ea22 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:08.463077 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 25/54] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:17:08 2022] GET /identity/v3/domains/default => generated 200 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:08.468739 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6422b60b-cd42-4a7b-b94f-16e55920ea22 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:08.495298 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:08.505091 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb672e2c-50dc-4fa9-a6b1-c37bd3b3a3f8 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:08.519173 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:08.539446 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c81dfc-e5b4-4db6-8c4d-3c2a21f512f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:08.540012 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c81dfc-e5b4-4db6-8c4d-3c2a21f512f7 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:08.540290 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c81dfc-e5b4-4db6-8c4d-3c2a21f512f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:08.540566 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c81dfc-e5b4-4db6-8c4d-3c2a21f512f7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:08.541692 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c81dfc-e5b4-4db6-8c4d-3c2a21f512f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:08.543165 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c81dfc-e5b4-4db6-8c4d-3c2a21f512f7 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:08.544717 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 30/55] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:17:08 2022] GET /identity/v3/domains/default => generated 200 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:17:08.550207 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c81dfc-e5b4-4db6-8c4d-3c2a21f512f7 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:08.569820 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6422b60b-cd42-4a7b-b94f-16e55920ea22 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:08.603742 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29404565-1232-4d17-99fd-b8e6843630b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:08.604380 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29404565-1232-4d17-99fd-b8e6843630b5 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:08.604651 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29404565-1232-4d17-99fd-b8e6843630b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:08.604929 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29404565-1232-4d17-99fd-b8e6843630b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:08.605693 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29404565-1232-4d17-99fd-b8e6843630b5 None admin] RBAC: Adding query filter params (role.id=36577a6f46d642d4b81805ede9872490, scope.domain.id=default, user.id=e6c9197e0f844432a03b9e75fc87712b) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:17:08.606003 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29404565-1232-4d17-99fd-b8e6843630b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:08.607461 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29404565-1232-4d17-99fd-b8e6843630b5 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:08.617152 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 26/56] 10.222.0.102 () {56 vars in 1327 bytes} [Thu Sep 1 17:17:08 2022] GET /identity/v3/role_assignments?user.id=e6c9197e0f844432a03b9e75fc87712b&scope.domain.id=default&role.id=36577a6f46d642d4b81805ede9872490 => generated 532 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:17:08.656542 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c81dfc-e5b4-4db6-8c4d-3c2a21f512f7 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:08.693899 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b097292-9c3e-4a83-9137-e2d56e5d3aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:08.694542 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b097292-9c3e-4a83-9137-e2d56e5d3aea None admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:08.695009 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b097292-9c3e-4a83-9137-e2d56e5d3aea None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:08.695282 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b097292-9c3e-4a83-9137-e2d56e5d3aea None admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:08.695948 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b097292-9c3e-4a83-9137-e2d56e5d3aea None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:08.697435 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b097292-9c3e-4a83-9137-e2d56e5d3aea None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:08.727386 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 31/57] 10.222.0.102 () {60 vars in 1145 bytes} [Thu Sep 1 17:17:08 2022] POST /identity/v3/projects => generated 299 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:17:10.628985 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dac7ea-513d-4b93-851f-9211a75183ed None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:10.629570 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dac7ea-513d-4b93-851f-9211a75183ed None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:10.629570 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dac7ea-513d-4b93-851f-9211a75183ed None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:10.630913 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 27/58] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:17:10 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:17:10.635982 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b42162-0934-4aec-a4af-12321485190b None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:10.636393 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b42162-0934-4aec-a4af-12321485190b None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:10.636549 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b42162-0934-4aec-a4af-12321485190b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:10.655398 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-a7b42162-0934-4aec-a4af-12321485190b None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:17:10.656588 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-a7b42162-0934-4aec-a4af-12321485190b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:17:10.664526 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:10.734421 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 32/59] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:17:10 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:17:11.012142 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dac7ea-513d-4b93-851f-9211a75183ed None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:11.113390 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dac7ea-513d-4b93-851f-9211a75183ed None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:11.127085 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:11.149662 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587b90f9-ab0d-48f2-9a29-99835e4d38ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:11.150303 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587b90f9-ab0d-48f2-9a29-99835e4d38ab None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:11.150593 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587b90f9-ab0d-48f2-9a29-99835e4d38ab None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:11.150879 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587b90f9-ab0d-48f2-9a29-99835e4d38ab None admin] PATH_INFO: `/v3/domains/default` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:11.152024 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587b90f9-ab0d-48f2-9a29-99835e4d38ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:11.153524 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587b90f9-ab0d-48f2-9a29-99835e4d38ab None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:11.155103 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 28/60] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:17:11 2022] GET /identity/v3/domains/default => generated 200 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:11.160729 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b42162-0934-4aec-a4af-12321485190b None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:11.259097 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b42162-0934-4aec-a4af-12321485190b None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:11.291596 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2260d8-ef6b-47bb-aca6-1586ff3b73f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:11.292142 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2260d8-ef6b-47bb-aca6-1586ff3b73f1 None admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:11.292429 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2260d8-ef6b-47bb-aca6-1586ff3b73f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:11.292708 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2260d8-ef6b-47bb-aca6-1586ff3b73f1 None admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:11.293381 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2260d8-ef6b-47bb-aca6-1586ff3b73f1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:11.294838 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2260d8-ef6b-47bb-aca6-1586ff3b73f1 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:11.317151 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:11.323609 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 33/61] 10.222.0.102 () {60 vars in 1137 bytes} [Thu Sep 1 17:17:11 2022] POST /identity/v3/users => generated 276 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:17:14.443398 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13a28ad-e9b2-49c3-80e4-ac7b6aec779a None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:14.444530 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13a28ad-e9b2-49c3-80e4-ac7b6aec779a None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:14.445013 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13a28ad-e9b2-49c3-80e4-ac7b6aec779a None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:14.446754 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 29/62] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:17:14 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:17:14.452396 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c423207-622f-4fa6-8619-48694d006885 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:14.452821 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c423207-622f-4fa6-8619-48694d006885 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:14.452956 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c423207-622f-4fa6-8619-48694d006885 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:14.477480 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-0c423207-622f-4fa6-8619-48694d006885 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:17:14.478681 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-0c423207-622f-4fa6-8619-48694d006885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:17:14.490969 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6263df8a-529c-4e97-a4af-817f4c3e0716 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:14.491256 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6263df8a-529c-4e97-a4af-817f4c3e0716 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:14.491532 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6263df8a-529c-4e97-a4af-817f4c3e0716 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:14.493203 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 30/63] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:17:14 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Sep 01 17:17:14.503487 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba97d122-9bb0-4297-be0b-bf01b1511ec4 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:14.503769 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba97d122-9bb0-4297-be0b-bf01b1511ec4 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:14.504179 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba97d122-9bb0-4297-be0b-bf01b1511ec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:14.540867 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-ba97d122-9bb0-4297-be0b-bf01b1511ec4 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:17:14.542100 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-ba97d122-9bb0-4297-be0b-bf01b1511ec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:17:14.583994 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 34/64] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:17:14 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:17:14.594651 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf90dd1-87d4-4b71-911a-bb82ab28fc1d None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:14.594936 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf90dd1-87d4-4b71-911a-bb82ab28fc1d None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:14.595209 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf90dd1-87d4-4b71-911a-bb82ab28fc1d None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:14.597208 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 35/65] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:17:14 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Sep 01 17:17:14.603285 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e290ec0a-e6af-4b12-b459-678cabb13944 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:14.603892 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e290ec0a-e6af-4b12-b459-678cabb13944 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:14.604359 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e290ec0a-e6af-4b12-b459-678cabb13944 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:14.635367 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-e290ec0a-e6af-4b12-b459-678cabb13944 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:17:14.638021 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-e290ec0a-e6af-4b12-b459-678cabb13944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:17:14.684126 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 31/66] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:17:14 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 17:17:14.763957 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 36/67] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:17:14 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Sep 01 17:17:14.904721 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba97d122-9bb0-4297-be0b-bf01b1511ec4 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:14.914726 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fe0c36-b7ed-4f35-8b57-12737485b885 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:14.915445 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fe0c36-b7ed-4f35-8b57-12737485b885 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:14.915445 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fe0c36-b7ed-4f35-8b57-12737485b885 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:14.916752 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 37/68] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:17:14 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Sep 01 17:17:14.934652 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31caf517-8f4c-486b-9ab4-a762cc8057b4 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:14.934908 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31caf517-8f4c-486b-9ab4-a762cc8057b4 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:14.935150 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31caf517-8f4c-486b-9ab4-a762cc8057b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:14.954172 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-31caf517-8f4c-486b-9ab4-a762cc8057b4 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:17:14.955397 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-31caf517-8f4c-486b-9ab4-a762cc8057b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:17:15.010388 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:15.029800 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 38/69] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:17:14 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 17:17:15.039764 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba97d122-9bb0-4297-be0b-bf01b1511ec4 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:15.051993 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31caf517-8f4c-486b-9ab4-a762cc8057b4 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:15.062409 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:15.110820 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6ac0f3-f8a6-4b48-a4fb-d71380a9011f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:15.116782 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6ac0f3-f8a6-4b48-a4fb-d71380a9011f None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:15.117298 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6ac0f3-f8a6-4b48-a4fb-d71380a9011f None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:15.117844 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6ac0f3-f8a6-4b48-a4fb-d71380a9011f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:15.131158 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6ac0f3-f8a6-4b48-a4fb-d71380a9011f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:17:15.138040 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-ee6ac0f3-f8a6-4b48-a4fb-d71380a9011f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:17:15.138040 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:17:15.138040 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:17:15.138040 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:17:15.138040 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:17:15.138040 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:17:15.138040 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:17:15.138040 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:17:15.138040 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:17:15.138040 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:17:15.138040 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:17:15.138040 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:17:15.138040 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:17:15.138040 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:17:15.138040 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:17:15.138040 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:17:15.138853 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:17:15.138853 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:17:15.138853 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:17:15.138853 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:17:15.138853 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:17:15.138853 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:17:15.138853 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:17:15.138853 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:17:15.138853 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:17:15.138853 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:17:15.138853 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:17:15.138853 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:17:15.138853 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:17:15.138853 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:17:15.138853 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:17:15.138853 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:17:15.139601 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:17:15.139601 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:17:15.139601 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:17:15.139601 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:17:15.139601 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:17:15.139601 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:17:15.139601 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:17:15.139601 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:17:15.139601 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:17:15.139601 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:17:15.139601 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:17:15.139601 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:17:15.141893 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 32/70] 10.222.0.102 () {56 vars in 1101 bytes} [Thu Sep 1 17:17:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:17:15.142869 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6ac0f3-f8a6-4b48-a4fb-d71380a9011f None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:15.167364 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31caf517-8f4c-486b-9ab4-a762cc8057b4 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:15.205166 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af30785f-00e7-4731-8671-3cd1152e002e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:15.205851 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af30785f-00e7-4731-8671-3cd1152e002e None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:15.206131 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af30785f-00e7-4731-8671-3cd1152e002e None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:15.206407 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af30785f-00e7-4731-8671-3cd1152e002e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:15.216935 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af30785f-00e7-4731-8671-3cd1152e002e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:17:15.221786 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-af30785f-00e7-4731-8671-3cd1152e002e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:17:15.221786 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:17:15.221786 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:17:15.221786 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:17:15.221786 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:17:15.221786 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:17:15.221786 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:17:15.221786 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:17:15.221786 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:17:15.221786 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:17:15.221786 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:17:15.221786 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:17:15.221786 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:17:15.221786 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:17:15.221786 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:17:15.221786 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:17:15.222669 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:17:15.222669 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:17:15.222669 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:17:15.222669 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:17:15.222669 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:17:15.222669 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:17:15.222669 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:17:15.222669 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:17:15.222669 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:17:15.222669 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:17:15.222669 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:17:15.222669 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:17:15.222669 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:17:15.222669 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:17:15.222669 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:17:15.222669 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:17:15.223418 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:17:15.223418 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:17:15.223418 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:17:15.223418 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:17:15.223418 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:17:15.223418 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:17:15.223418 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:17:15.223418 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:17:15.223418 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:17:15.223418 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:17:15.223418 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:17:15.223418 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:17:15.223418 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 39/71] 10.222.0.102 () {56 vars in 1104 bytes} [Thu Sep 1 17:17:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:17:15.229775 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3704308e-30ab-4d78-9510-44d4fd59c753 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:15.229775 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3704308e-30ab-4d78-9510-44d4fd59c753 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:15.229775 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3704308e-30ab-4d78-9510-44d4fd59c753 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:15.229775 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 40/72] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:17:15 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:17:15.229775 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3704308e-30ab-4d78-9510-44d4fd59c753 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:15.264004 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6ac0f3-f8a6-4b48-a4fb-d71380a9011f None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:15.284383 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:15.302927 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048a2711-7e4a-439d-b44e-0bafb2ae7ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:15.303558 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048a2711-7e4a-439d-b44e-0bafb2ae7ba4 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:15.303841 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048a2711-7e4a-439d-b44e-0bafb2ae7ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:15.304120 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048a2711-7e4a-439d-b44e-0bafb2ae7ba4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:15.318200 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048a2711-7e4a-439d-b44e-0bafb2ae7ba4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:17:15.323152 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-048a2711-7e4a-439d-b44e-0bafb2ae7ba4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 17:17:15.323152 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:17:15.323152 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:17:15.323152 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:17:15.323152 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:17:15.323152 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:17:15.323152 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:17:15.323152 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:17:15.323152 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:17:15.323152 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:17:15.323152 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:17:15.323152 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:17:15.323152 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:17:15.323152 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:17:15.323152 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:17:15.323152 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:17:15.324153 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:17:15.324153 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:17:15.324153 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:17:15.324153 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:17:15.324153 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:17:15.324153 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:17:15.324153 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:17:15.324153 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:17:15.324153 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:17:15.324153 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:17:15.324153 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:17:15.324153 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:17:15.324153 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:17:15.324153 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:17:15.324153 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:17:15.324153 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:17:15.324985 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:17:15.324985 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:17:15.324985 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:17:15.324985 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:17:15.324985 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:17:15.324985 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:17:15.324985 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:17:15.324985 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:17:15.324985 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:17:15.324985 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:17:15.324985 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 17:17:15.324985 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:17:15.324985 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 33/73] 10.222.0.102 () {56 vars in 1119 bytes} [Thu Sep 1 17:17:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:17:15.325772 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048a2711-7e4a-439d-b44e-0bafb2ae7ba4 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:15.342892 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3704308e-30ab-4d78-9510-44d4fd59c753 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:15.357290 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:15.378206 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cd439a-f460-4346-8590-7e6eef62dfea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:15.378813 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cd439a-f460-4346-8590-7e6eef62dfea None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:15.379687 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cd439a-f460-4346-8590-7e6eef62dfea None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:15.379965 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cd439a-f460-4346-8590-7e6eef62dfea None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:15.380621 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cd439a-f460-4346-8590-7e6eef62dfea None admin] RBAC: Adding query filter params (name=admin) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:17:15.380929 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cd439a-f460-4346-8590-7e6eef62dfea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:15.382412 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cd439a-f460-4346-8590-7e6eef62dfea None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:15.390787 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 41/74] 10.222.0.102 () {56 vars in 1104 bytes} [Thu Sep 1 17:17:15 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:15.394670 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863afa4c-6689-4ff4-89c1-ab5c209ae447 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:15.394965 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863afa4c-6689-4ff4-89c1-ab5c209ae447 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:15.395251 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863afa4c-6689-4ff4-89c1-ab5c209ae447 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:15.413443 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-863afa4c-6689-4ff4-89c1-ab5c209ae447 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:17:15.414679 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-863afa4c-6689-4ff4-89c1-ab5c209ae447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:17:15.433048 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048a2711-7e4a-439d-b44e-0bafb2ae7ba4 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:15.468132 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403c960a-8d3a-470a-a0e4-dc3718e19e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:15.468778 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403c960a-8d3a-470a-a0e4-dc3718e19e67 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:15.469001 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403c960a-8d3a-470a-a0e4-dc3718e19e67 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:15.469272 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403c960a-8d3a-470a-a0e4-dc3718e19e67 None admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:15.469957 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403c960a-8d3a-470a-a0e4-dc3718e19e67 None admin] RBAC: Adding query filter params (name=member) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:17:15.470265 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403c960a-8d3a-470a-a0e4-dc3718e19e67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:15.471648 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403c960a-8d3a-470a-a0e4-dc3718e19e67 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:15.479760 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 34/75] 10.222.0.102 () {56 vars in 1106 bytes} [Thu Sep 1 17:17:15 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:15.480637 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403c960a-8d3a-470a-a0e4-dc3718e19e67 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:15.483795 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 42/76] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:17:15 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:17:15.484632 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863afa4c-6689-4ff4-89c1-ab5c209ae447 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:15.526736 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:15.577604 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:15.588722 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403c960a-8d3a-470a-a0e4-dc3718e19e67 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:15.590449 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863afa4c-6689-4ff4-89c1-ab5c209ae447 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:15.623769 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935a1974-975d-4d38-8201-e43e43126047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:15.624382 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935a1974-975d-4d38-8201-e43e43126047 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:15.624667 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935a1974-975d-4d38-8201-e43e43126047 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:15.624969 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935a1974-975d-4d38-8201-e43e43126047 None admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:15.625655 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935a1974-975d-4d38-8201-e43e43126047 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:17:15.625959 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935a1974-975d-4d38-8201-e43e43126047 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:15.627328 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935a1974-975d-4d38-8201-e43e43126047 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:15.630333 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad62f94a-eba9-4b20-9818-851a076a0446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:15.631152 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad62f94a-eba9-4b20-9818-851a076a0446 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:15.631600 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad62f94a-eba9-4b20-9818-851a076a0446 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:15.632045 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad62f94a-eba9-4b20-9818-851a076a0446 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:15.636435 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 35/77] 10.222.0.102 () {56 vars in 1116 bytes} [Thu Sep 1 17:17:15 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:15.637410 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935a1974-975d-4d38-8201-e43e43126047 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:15.643314 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad62f94a-eba9-4b20-9818-851a076a0446 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:17:15.649007 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-ad62f94a-eba9-4b20-9818-851a076a0446 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:17:15.649007 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:17:15.649007 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:17:15.649007 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:17:15.649007 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:17:15.649007 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:17:15.649007 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:17:15.649007 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:17:15.649007 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:17:15.649007 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:17:15.649007 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:17:15.649007 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:17:15.649007 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:17:15.649007 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:17:15.649007 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:17:15.649007 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:17:15.650073 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:17:15.650073 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:17:15.650073 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:17:15.650073 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:17:15.650073 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:17:15.650073 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:17:15.650073 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:17:15.650073 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:17:15.650073 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:17:15.650073 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:17:15.650073 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:17:15.650073 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:17:15.650073 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:17:15.650073 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:17:15.650073 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:17:15.650073 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:17:15.650998 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:17:15.650998 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:17:15.650998 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:17:15.650998 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:17:15.650998 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:17:15.650998 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:17:15.650998 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:17:15.650998 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:17:15.650998 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:17:15.650998 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:17:15.650998 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:17:15.650998 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:17:15.653413 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 43/78] 10.222.0.102 () {56 vars in 1104 bytes} [Thu Sep 1 17:17:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:17:15.654353 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad62f94a-eba9-4b20-9818-851a076a0446 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:15.754879 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935a1974-975d-4d38-8201-e43e43126047 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:15.761502 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:15.764474 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad62f94a-eba9-4b20-9818-851a076a0446 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:15.789046 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b23860-5d1b-404a-97e9-1f4d064ffbf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:15.789705 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b23860-5d1b-404a-97e9-1f4d064ffbf2 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:15.789987 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b23860-5d1b-404a-97e9-1f4d064ffbf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:15.790264 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b23860-5d1b-404a-97e9-1f4d064ffbf2 None admin] PATH_INFO: `/v3/users/e6c9197e0f844432a03b9e75fc87712b` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:15.791500 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b23860-5d1b-404a-97e9-1f4d064ffbf2 None admin] RBAC: Authorizing `identity:get_user(user_id=e6c9197e0f844432a03b9e75fc87712b)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:15.792837 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b23860-5d1b-404a-97e9-1f4d064ffbf2 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:15.794439 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 36/79] 10.222.0.102 () {56 vars in 1182 bytes} [Thu Sep 1 17:17:15 2022] GET /identity/v3/users/e6c9197e0f844432a03b9e75fc87712b => generated 248 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:15.795209 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b23860-5d1b-404a-97e9-1f4d064ffbf2 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:15.800181 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2845af9e-4894-49d6-8a51-6b869ec33e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:15.800853 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2845af9e-4894-49d6-8a51-6b869ec33e82 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:15.801132 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2845af9e-4894-49d6-8a51-6b869ec33e82 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:15.801440 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2845af9e-4894-49d6-8a51-6b869ec33e82 None admin] PATH_INFO: `/v3/users/27f309e66b2c4003a820573ada1b0958` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:15.814955 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2845af9e-4894-49d6-8a51-6b869ec33e82 None admin] RBAC: Authorizing `identity:get_user(user_id=27f309e66b2c4003a820573ada1b0958)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:15.816496 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2845af9e-4894-49d6-8a51-6b869ec33e82 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:15.818267 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 44/80] 10.222.0.102 () {56 vars in 1182 bytes} [Thu Sep 1 17:17:15 2022] GET /identity/v3/users/27f309e66b2c4003a820573ada1b0958 => generated 276 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:15.819356 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2845af9e-4894-49d6-8a51-6b869ec33e82 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:15.822354 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:15.903917 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b23860-5d1b-404a-97e9-1f4d064ffbf2 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:15.931172 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2845af9e-4894-49d6-8a51-6b869ec33e82 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:15.937184 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fea6e4-ef4c-412b-b29a-49b6040e83bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:15.937848 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fea6e4-ef4c-412b-b29a-49b6040e83bd None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:15.938131 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fea6e4-ef4c-412b-b29a-49b6040e83bd None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:15.938408 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fea6e4-ef4c-412b-b29a-49b6040e83bd None admin] PATH_INFO: `/v3/users/27f309e66b2c4003a820573ada1b0958` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:15.939544 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fea6e4-ef4c-412b-b29a-49b6040e83bd None admin] RBAC: Authorizing `identity:get_user(user_id=27f309e66b2c4003a820573ada1b0958)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:15.940983 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fea6e4-ef4c-412b-b29a-49b6040e83bd None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:15.942634 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 37/81] 10.222.0.102 () {56 vars in 1182 bytes} [Thu Sep 1 17:17:15 2022] GET /identity/v3/users/27f309e66b2c4003a820573ada1b0958 => generated 276 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:15.943400 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fea6e4-ef4c-412b-b29a-49b6040e83bd None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:15.970834 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d07328f-cb31-43ce-a36f-eba93c248e98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:15.971454 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d07328f-cb31-43ce-a36f-eba93c248e98 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:15.971739 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d07328f-cb31-43ce-a36f-eba93c248e98 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:15.972015 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d07328f-cb31-43ce-a36f-eba93c248e98 None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:15.972680 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d07328f-cb31-43ce-a36f-eba93c248e98 None admin] RBAC: Adding query filter params (name=member) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:17:15.972989 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d07328f-cb31-43ce-a36f-eba93c248e98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:15.974984 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d07328f-cb31-43ce-a36f-eba93c248e98 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:15.983948 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 45/82] 10.222.0.102 () {56 vars in 1106 bytes} [Thu Sep 1 17:17:15 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:15.984751 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d07328f-cb31-43ce-a36f-eba93c248e98 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:16.006086 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:16.038005 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:16.048421 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fea6e4-ef4c-412b-b29a-49b6040e83bd None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:16.081696 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd10a95-1ec7-49cb-96ce-5e3fdd6228f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:16.082316 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd10a95-1ec7-49cb-96ce-5e3fdd6228f7 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:16.082595 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd10a95-1ec7-49cb-96ce-5e3fdd6228f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:16.082869 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd10a95-1ec7-49cb-96ce-5e3fdd6228f7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:16.084055 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd10a95-1ec7-49cb-96ce-5e3fdd6228f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:16.085524 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd10a95-1ec7-49cb-96ce-5e3fdd6228f7 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:16.087095 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 38/83] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:17:15 2022] GET /identity/v3/domains/default => generated 200 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:16.087860 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd10a95-1ec7-49cb-96ce-5e3fdd6228f7 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:16.088285 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d07328f-cb31-43ce-a36f-eba93c248e98 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:16.123173 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e01905-86b8-4d32-b5af-d0c8a59508b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:16.123789 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e01905-86b8-4d32-b5af-d0c8a59508b6 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:16.124073 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e01905-86b8-4d32-b5af-d0c8a59508b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:16.124353 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e01905-86b8-4d32-b5af-d0c8a59508b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:16.189212 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd10a95-1ec7-49cb-96ce-5e3fdd6228f7 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:16.191361 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e01905-86b8-4d32-b5af-d0c8a59508b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:16.192888 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e01905-86b8-4d32-b5af-d0c8a59508b6 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:16.213294 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:16.222767 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca92bb28-220c-4a83-b53a-ac69fd9ec2bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:16.223377 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca92bb28-220c-4a83-b53a-ac69fd9ec2bf None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:16.223657 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca92bb28-220c-4a83-b53a-ac69fd9ec2bf None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:16.223927 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca92bb28-220c-4a83-b53a-ac69fd9ec2bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:16.251679 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:16.292701 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca92bb28-220c-4a83-b53a-ac69fd9ec2bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:16.292873 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 46/84] 10.222.0.102 () {58 vars in 1287 bytes} [Thu Sep 1 17:17:15 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 17:17:16.293588 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e01905-86b8-4d32-b5af-d0c8a59508b6 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:16.294210 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca92bb28-220c-4a83-b53a-ac69fd9ec2bf None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:16.395969 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 39/85] 10.222.0.102 () {58 vars in 1287 bytes} [Thu Sep 1 17:17:16 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 17:17:16.396569 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca92bb28-220c-4a83-b53a-ac69fd9ec2bf None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:16.407611 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e01905-86b8-4d32-b5af-d0c8a59508b6 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:16.428140 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:16.443109 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ba5a6d-d19d-4c5f-a1bc-d18eb0649395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:16.443649 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ba5a6d-d19d-4c5f-a1bc-d18eb0649395 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:16.443931 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ba5a6d-d19d-4c5f-a1bc-d18eb0649395 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:16.444207 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ba5a6d-d19d-4c5f-a1bc-d18eb0649395 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:16.457204 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:16.506748 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca92bb28-220c-4a83-b53a-ac69fd9ec2bf None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:16.514385 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ba5a6d-d19d-4c5f-a1bc-d18eb0649395 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:16.515895 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ba5a6d-d19d-4c5f-a1bc-d18eb0649395 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:16.542269 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cac4fd-a5f9-4277-93cb-e21e02fcbe8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:16.542900 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cac4fd-a5f9-4277-93cb-e21e02fcbe8c None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:16.543191 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cac4fd-a5f9-4277-93cb-e21e02fcbe8c None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:16.543462 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cac4fd-a5f9-4277-93cb-e21e02fcbe8c None admin] PATH_INFO: `/v3/users/27f309e66b2c4003a820573ada1b0958` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:16.544595 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cac4fd-a5f9-4277-93cb-e21e02fcbe8c None admin] RBAC: Authorizing `identity:get_user(user_id=27f309e66b2c4003a820573ada1b0958)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:16.546090 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cac4fd-a5f9-4277-93cb-e21e02fcbe8c None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:16.547687 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 40/86] 10.222.0.102 () {56 vars in 1182 bytes} [Thu Sep 1 17:17:16 2022] GET /identity/v3/users/27f309e66b2c4003a820573ada1b0958 => generated 276 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:16.548568 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cac4fd-a5f9-4277-93cb-e21e02fcbe8c None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:16.618027 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 47/87] 10.222.0.102 () {58 vars in 1287 bytes} [Thu Sep 1 17:17:16 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 17:17:16.618875 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ba5a6d-d19d-4c5f-a1bc-d18eb0649395 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:16.641203 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:16.651265 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cac4fd-a5f9-4277-93cb-e21e02fcbe8c None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:16.676971 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:16.685817 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461b322e-0586-498a-b987-6fe58d934a8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:16.686419 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461b322e-0586-498a-b987-6fe58d934a8d None admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:16.686707 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461b322e-0586-498a-b987-6fe58d934a8d None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:16.686974 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461b322e-0586-498a-b987-6fe58d934a8d None admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:16.687623 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461b322e-0586-498a-b987-6fe58d934a8d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:16.689049 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461b322e-0586-498a-b987-6fe58d934a8d None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:16.716147 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 41/88] 10.222.0.102 () {60 vars in 1137 bytes} [Thu Sep 1 17:17:16 2022] POST /identity/v3/users => generated 290 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:17:16.716720 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461b322e-0586-498a-b987-6fe58d934a8d None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:16.730415 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ba5a6d-d19d-4c5f-a1bc-d18eb0649395 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:16.773049 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6b661c-71b8-49ee-baeb-fbe8fea00d9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:16.773796 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6b661c-71b8-49ee-baeb-fbe8fea00d9b None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:16.774066 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6b661c-71b8-49ee-baeb-fbe8fea00d9b None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:16.774345 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6b661c-71b8-49ee-baeb-fbe8fea00d9b None admin] PATH_INFO: `/v3/projects/8e88c67b942f45108c95a833547f9bc8` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:16.775541 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6b661c-71b8-49ee-baeb-fbe8fea00d9b None admin] RBAC: Authorizing `identity:get_project(project_id=8e88c67b942f45108c95a833547f9bc8)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:16.777058 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6b661c-71b8-49ee-baeb-fbe8fea00d9b None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:16.778994 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 48/89] 10.222.0.102 () {56 vars in 1191 bytes} [Thu Sep 1 17:17:16 2022] GET /identity/v3/projects/8e88c67b942f45108c95a833547f9bc8 => generated 299 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:16.779640 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6b661c-71b8-49ee-baeb-fbe8fea00d9b None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:16.838847 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461b322e-0586-498a-b987-6fe58d934a8d None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:16.887524 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e81d48-cf12-4b94-aed3-60fd40f78421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:16.888304 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e81d48-cf12-4b94-aed3-60fd40f78421 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:16.888659 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e81d48-cf12-4b94-aed3-60fd40f78421 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:16.889010 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e81d48-cf12-4b94-aed3-60fd40f78421 None admin] PATH_INFO: `/v3/projects/8e88c67b942f45108c95a833547f9bc8` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:16.889453 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:16.891409 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e81d48-cf12-4b94-aed3-60fd40f78421 None admin] RBAC: Authorizing `identity:get_project(project_id=8e88c67b942f45108c95a833547f9bc8)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:16.893099 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e81d48-cf12-4b94-aed3-60fd40f78421 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:16.895022 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 42/90] 10.222.0.102 () {56 vars in 1191 bytes} [Thu Sep 1 17:17:16 2022] GET /identity/v3/projects/8e88c67b942f45108c95a833547f9bc8 => generated 299 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:16.896749 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e81d48-cf12-4b94-aed3-60fd40f78421 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:16.908425 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6b661c-71b8-49ee-baeb-fbe8fea00d9b None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:16.955494 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:16.956474 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bbf27a-767c-491f-98c4-123f00817262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:16.957148 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bbf27a-767c-491f-98c4-123f00817262 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:16.957500 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bbf27a-767c-491f-98c4-123f00817262 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:16.957786 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bbf27a-767c-491f-98c4-123f00817262 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:17.022606 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e81d48-cf12-4b94-aed3-60fd40f78421 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:17.047661 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bbf27a-767c-491f-98c4-123f00817262 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:17.049359 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bbf27a-767c-491f-98c4-123f00817262 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:17.065248 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5e0fb7-7973-44c1-b8cb-9717876c477e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:17.065924 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5e0fb7-7973-44c1-b8cb-9717876c477e None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:17.066209 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5e0fb7-7973-44c1-b8cb-9717876c477e None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:17.066524 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5e0fb7-7973-44c1-b8cb-9717876c477e None admin] PATH_INFO: `/v3/projects/8e88c67b942f45108c95a833547f9bc8` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:17.067716 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5e0fb7-7973-44c1-b8cb-9717876c477e None admin] RBAC: Authorizing `identity:get_project(project_id=8e88c67b942f45108c95a833547f9bc8)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:17.069180 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5e0fb7-7973-44c1-b8cb-9717876c477e None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:17.071609 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 43/91] 10.222.0.102 () {56 vars in 1191 bytes} [Thu Sep 1 17:17:16 2022] GET /identity/v3/projects/8e88c67b942f45108c95a833547f9bc8 => generated 299 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:17.073059 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5e0fb7-7973-44c1-b8cb-9717876c477e None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:17.158852 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:17.184235 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 49/92] 10.222.0.102 () {58 vars in 1287 bytes} [Thu Sep 1 17:17:16 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 17:17:17.185465 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bbf27a-767c-491f-98c4-123f00817262 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:17.205203 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5e0fb7-7973-44c1-b8cb-9717876c477e None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:17.225147 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:17.243562 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02392214-1a85-4634-bbff-292473b2bd1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:17.244178 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02392214-1a85-4634-bbff-292473b2bd1e None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:17.244465 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02392214-1a85-4634-bbff-292473b2bd1e None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:17.244740 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02392214-1a85-4634-bbff-292473b2bd1e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:17.245503 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02392214-1a85-4634-bbff-292473b2bd1e None admin] RBAC: Adding query filter params (role.id=36577a6f46d642d4b81805ede9872490, scope.project.id=8e88c67b942f45108c95a833547f9bc8, user.id=e6c9197e0f844432a03b9e75fc87712b) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:17:17.245815 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02392214-1a85-4634-bbff-292473b2bd1e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:17.247277 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02392214-1a85-4634-bbff-292473b2bd1e None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:17.253534 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 44/93] 10.222.0.102 () {56 vars in 1379 bytes} [Thu Sep 1 17:17:17 2022] GET /identity/v3/role_assignments?user.id=e6c9197e0f844432a03b9e75fc87712b&scope.project.id=8e88c67b942f45108c95a833547f9bc8&role.id=36577a6f46d642d4b81805ede9872490 => generated 260 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:17.254545 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02392214-1a85-4634-bbff-292473b2bd1e None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:17.339493 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bbf27a-767c-491f-98c4-123f00817262 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:17.370270 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02392214-1a85-4634-bbff-292473b2bd1e None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:17.397174 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cec0e5d-52d6-400f-a18e-19fa97e8694b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:17.402232 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cec0e5d-52d6-400f-a18e-19fa97e8694b None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:17.402515 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cec0e5d-52d6-400f-a18e-19fa97e8694b None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:17.403843 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cec0e5d-52d6-400f-a18e-19fa97e8694b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:17.404604 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cec0e5d-52d6-400f-a18e-19fa97e8694b None admin] RBAC: Adding query filter params (role.id=b75cb360925e4ba5a65f14965c103ffd, scope.project.id=8e88c67b942f45108c95a833547f9bc8, user.id=27f309e66b2c4003a820573ada1b0958) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:17:17.404925 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cec0e5d-52d6-400f-a18e-19fa97e8694b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:17.406497 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cec0e5d-52d6-400f-a18e-19fa97e8694b None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:17.418520 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 50/94] 10.222.0.102 () {56 vars in 1379 bytes} [Thu Sep 1 17:17:17 2022] GET /identity/v3/role_assignments?user.id=27f309e66b2c4003a820573ada1b0958&scope.project.id=8e88c67b942f45108c95a833547f9bc8&role.id=b75cb360925e4ba5a65f14965c103ffd => generated 260 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:17.420895 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cec0e5d-52d6-400f-a18e-19fa97e8694b None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:17.433861 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ebc62a-97f6-43b3-abf6-d2085a8c2cc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:17.434498 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ebc62a-97f6-43b3-abf6-d2085a8c2cc1 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:17.434785 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ebc62a-97f6-43b3-abf6-d2085a8c2cc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:17.435113 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ebc62a-97f6-43b3-abf6-d2085a8c2cc1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:17.435874 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ebc62a-97f6-43b3-abf6-d2085a8c2cc1 None admin] RBAC: Adding query filter params (role.id=abef78c185c74c16a2c7e8a170583f9f, scope.project.id=8e88c67b942f45108c95a833547f9bc8, user.id=27f309e66b2c4003a820573ada1b0958) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:17:17.436187 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ebc62a-97f6-43b3-abf6-d2085a8c2cc1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:17.437693 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ebc62a-97f6-43b3-abf6-d2085a8c2cc1 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:17.453165 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 45/95] 10.222.0.102 () {56 vars in 1379 bytes} [Thu Sep 1 17:17:17 2022] GET /identity/v3/role_assignments?user.id=27f309e66b2c4003a820573ada1b0958&scope.project.id=8e88c67b942f45108c95a833547f9bc8&role.id=abef78c185c74c16a2c7e8a170583f9f => generated 260 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:17.505781 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:17.561998 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cec0e5d-52d6-400f-a18e-19fa97e8694b None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:17.608765 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5125578-ce7d-49cc-8dc7-693a54e67e07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:17.609379 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5125578-ce7d-49cc-8dc7-693a54e67e07 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:17.609670 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5125578-ce7d-49cc-8dc7-693a54e67e07 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:17.609953 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5125578-ce7d-49cc-8dc7-693a54e67e07 None admin] PATH_INFO: `/v3/projects/dc1ae957803645b4abe7503e4399bf10` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:17.611275 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5125578-ce7d-49cc-8dc7-693a54e67e07 None admin] RBAC: Authorizing `identity:get_project(project_id=dc1ae957803645b4abe7503e4399bf10)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:17.612900 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5125578-ce7d-49cc-8dc7-693a54e67e07 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:17.615333 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 51/96] 10.222.0.102 () {56 vars in 1191 bytes} [Thu Sep 1 17:17:17 2022] GET /identity/v3/projects/dc1ae957803645b4abe7503e4399bf10 => generated 313 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:17.621702 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ebc62a-97f6-43b3-abf6-d2085a8c2cc1 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:17.674780 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:17.743272 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ebc62a-97f6-43b3-abf6-d2085a8c2cc1 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:17.781731 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b82efd5-3622-4118-b63a-d02a91b710b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:17.782353 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b82efd5-3622-4118-b63a-d02a91b710b7 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:17.782637 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b82efd5-3622-4118-b63a-d02a91b710b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:17.782913 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b82efd5-3622-4118-b63a-d02a91b710b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:17.783651 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b82efd5-3622-4118-b63a-d02a91b710b7 None admin] RBAC: Adding query filter params (role.id=b75cb360925e4ba5a65f14965c103ffd, scope.project.id=dc1ae957803645b4abe7503e4399bf10, user.id=27f309e66b2c4003a820573ada1b0958) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:17:17.783965 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b82efd5-3622-4118-b63a-d02a91b710b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:17.785957 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b82efd5-3622-4118-b63a-d02a91b710b7 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:17.792734 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 46/97] 10.222.0.102 () {56 vars in 1379 bytes} [Thu Sep 1 17:17:17 2022] GET /identity/v3/role_assignments?user.id=27f309e66b2c4003a820573ada1b0958&scope.project.id=dc1ae957803645b4abe7503e4399bf10&role.id=b75cb360925e4ba5a65f14965c103ffd => generated 260 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:19.327609 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf18d96-d96c-4ace-99ba-48c99f6afcb3 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:19.328134 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf18d96-d96c-4ace-99ba-48c99f6afcb3 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:19.328206 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf18d96-d96c-4ace-99ba-48c99f6afcb3 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:19.329610 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 52/98] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:17:19 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:17:19.335465 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00000806-3233-41aa-af5c-d830d786b81a None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:19.336095 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00000806-3233-41aa-af5c-d830d786b81a None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:19.336436 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00000806-3233-41aa-af5c-d830d786b81a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:19.361402 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-00000806-3233-41aa-af5c-d830d786b81a None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:17:19.362587 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-00000806-3233-41aa-af5c-d830d786b81a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:17:19.444617 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38cfa32-706a-446c-9af1-fe81924802fc None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:19.444617 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38cfa32-706a-446c-9af1-fe81924802fc None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:19.444617 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38cfa32-706a-446c-9af1-fe81924802fc None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:19.444617 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 53/99] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:17:19 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:17:19.452077 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3495713a-b7f3-42cd-89af-b956db35379a None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:19.452656 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3495713a-b7f3-42cd-89af-b956db35379a None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:19.453159 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3495713a-b7f3-42cd-89af-b956db35379a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:19.453730 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 47/100] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:17:19 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:17:19.495345 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-3495713a-b7f3-42cd-89af-b956db35379a None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:17:19.496903 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-3495713a-b7f3-42cd-89af-b956db35379a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:17:19.598660 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 54/101] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:17:19 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 17:17:19.994882 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00000806-3233-41aa-af5c-d830d786b81a None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:20.017381 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3495713a-b7f3-42cd-89af-b956db35379a None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:20.047565 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:20.107923 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:20.192255 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00000806-3233-41aa-af5c-d830d786b81a None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:20.196115 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3495713a-b7f3-42cd-89af-b956db35379a None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:20.246969 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36300cb2-5bc6-4f96-ab97-874092c0cc7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:20.247893 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36300cb2-5bc6-4f96-ab97-874092c0cc7d None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:20.248462 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36300cb2-5bc6-4f96-ab97-874092c0cc7d None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:20.248844 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0dca2d-a61a-48ea-b35a-37ce5522df87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:20.248983 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36300cb2-5bc6-4f96-ab97-874092c0cc7d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:20.249915 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0dca2d-a61a-48ea-b35a-37ce5522df87 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:20.250453 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0dca2d-a61a-48ea-b35a-37ce5522df87 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:20.250922 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0dca2d-a61a-48ea-b35a-37ce5522df87 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:20.254002 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36300cb2-5bc6-4f96-ab97-874092c0cc7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:20.255653 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36300cb2-5bc6-4f96-ab97-874092c0cc7d None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:20.259833 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 48/102] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:17:19 2022] GET /identity/v3/domains/default => generated 200 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:20.267986 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3021834b-f8b6-44e1-a150-add64ade4d23 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:20.268765 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3021834b-f8b6-44e1-a150-add64ade4d23 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:20.270477 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3021834b-f8b6-44e1-a150-add64ade4d23 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:20.274242 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 49/103] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:17:20 2022] GET /identity => generated 271 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:17:20.275174 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3021834b-f8b6-44e1-a150-add64ade4d23 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:20.282663 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0dca2d-a61a-48ea-b35a-37ce5522df87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:17:20.290512 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-0e0dca2d-a61a-48ea-b35a-37ce5522df87 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 17:17:20.290512 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:17:20.290512 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:17:20.290512 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:17:20.290512 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:17:20.290512 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:17:20.290512 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:17:20.290512 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:17:20.290512 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:17:20.290512 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:17:20.290512 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:17:20.290512 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:17:20.290512 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:17:20.290512 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:17:20.290512 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:17:20.290512 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:17:20.291345 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:17:20.291345 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:17:20.291345 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:17:20.291345 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:17:20.291345 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:17:20.291345 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:17:20.291345 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:17:20.291345 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:17:20.291345 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:17:20.291345 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:17:20.291345 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:17:20.291345 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:17:20.291345 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:17:20.291345 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:17:20.291345 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:17:20.291345 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:17:20.292096 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:17:20.292096 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:17:20.292096 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:17:20.292096 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:17:20.292096 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:17:20.292096 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:17:20.292096 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:17:20.292096 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:17:20.292096 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:17:20.292096 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:17:20.292096 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 17:17:20.292096 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:17:20.302018 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 55/104] 10.222.0.102 () {56 vars in 1104 bytes} [Thu Sep 1 17:17:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:17:20.303626 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706a9a92-759c-4726-91dd-ce34d64a90b7 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:20.305443 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706a9a92-759c-4726-91dd-ce34d64a90b7 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:20.306638 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706a9a92-759c-4726-91dd-ce34d64a90b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:20.333113 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-706a9a92-759c-4726-91dd-ce34d64a90b7 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:17:20.335152 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-706a9a92-759c-4726-91dd-ce34d64a90b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:17:20.476660 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 56/105] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:17:20 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:17:20.482686 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706a9a92-759c-4726-91dd-ce34d64a90b7 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:20.553199 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3021834b-f8b6-44e1-a150-add64ade4d23 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:20.564951 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:20.582901 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:20.599709 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa3ccd3-89b9-48de-8f40-d3023c929d0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:20.600322 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa3ccd3-89b9-48de-8f40-d3023c929d0c None admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:20.600605 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa3ccd3-89b9-48de-8f40-d3023c929d0c None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:20.600880 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa3ccd3-89b9-48de-8f40-d3023c929d0c None admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:20.601834 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa3ccd3-89b9-48de-8f40-d3023c929d0c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:20.603287 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa3ccd3-89b9-48de-8f40-d3023c929d0c None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:20.645336 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 50/106] 10.222.0.102 () {60 vars in 1146 bytes} [Thu Sep 1 17:17:20 2022] POST /identity/v3/projects => generated 303 bytes in 365 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:17:20.645336 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24a873c-e597-46bb-89ed-db9581dd666a None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:20.645336 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24a873c-e597-46bb-89ed-db9581dd666a None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:20.645336 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24a873c-e597-46bb-89ed-db9581dd666a None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:20.645336 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 51/107] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:17:20 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:17:20.645336 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e2f152-8f9c-4913-9c6d-a3af10a00b5c None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:20.646082 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e2f152-8f9c-4913-9c6d-a3af10a00b5c None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:20.646082 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e2f152-8f9c-4913-9c6d-a3af10a00b5c None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:20.647010 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 52/108] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:17:20 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:17:20.657349 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1812ab-f14e-4729-b584-09c85ce2236e None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:20.657349 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1812ab-f14e-4729-b584-09c85ce2236e None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:20.657349 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1812ab-f14e-4729-b584-09c85ce2236e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:20.694255 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-ff1812ab-f14e-4729-b584-09c85ce2236e None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:17:20.695483 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-ff1812ab-f14e-4729-b584-09c85ce2236e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:17:20.731357 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706a9a92-759c-4726-91dd-ce34d64a90b7 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:20.784264 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 53/109] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:17:20 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:17:20.785864 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0704067-441d-4d5c-bf1f-fefd5c43b19e None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:20.786154 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0704067-441d-4d5c-bf1f-fefd5c43b19e None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:20.786483 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0704067-441d-4d5c-bf1f-fefd5c43b19e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:20.805544 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-b0704067-441d-4d5c-bf1f-fefd5c43b19e None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:17:20.806658 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-b0704067-441d-4d5c-bf1f-fefd5c43b19e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:17:20.844868 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f2a36a-45f0-4889-aef6-d2d3ee58157a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:20.845533 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f2a36a-45f0-4889-aef6-d2d3ee58157a None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:20.845819 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f2a36a-45f0-4889-aef6-d2d3ee58157a None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:20.846095 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f2a36a-45f0-4889-aef6-d2d3ee58157a None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:20.846777 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f2a36a-45f0-4889-aef6-d2d3ee58157a None admin] RBAC: Adding query filter params (name=reader) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:17:20.852438 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f2a36a-45f0-4889-aef6-d2d3ee58157a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:20.862218 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f2a36a-45f0-4889-aef6-d2d3ee58157a None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:20.880720 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 57/110] 10.222.0.102 () {56 vars in 1106 bytes} [Thu Sep 1 17:17:20 2022] GET /identity/v3/roles?name=reader => generated 343 bytes in 403 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:20.885964 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f2a36a-45f0-4889-aef6-d2d3ee58157a None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:20.886282 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 54/111] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:17:20 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:17:20.895672 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98d9643-d8e9-403e-a21c-0b551341f829 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:20.896032 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98d9643-d8e9-403e-a21c-0b551341f829 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:20.896228 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98d9643-d8e9-403e-a21c-0b551341f829 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:20.897724 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 55/112] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:17:20 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Sep 01 17:17:20.906458 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e0131f-0ca5-4033-a95c-bcd3f30057b6 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:20.906735 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e0131f-0ca5-4033-a95c-bcd3f30057b6 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:20.907014 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e0131f-0ca5-4033-a95c-bcd3f30057b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:20.939701 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-47e0131f-0ca5-4033-a95c-bcd3f30057b6 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:17:20.940904 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-47e0131f-0ca5-4033-a95c-bcd3f30057b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:17:20.972890 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:21.046009 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 56/113] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:17:20 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:17:21.047320 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e0131f-0ca5-4033-a95c-bcd3f30057b6 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:21.049738 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f2a36a-45f0-4889-aef6-d2d3ee58157a None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:21.077756 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:21.092010 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46093513-c920-4dc4-8525-53ba19eb3bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:21.093039 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46093513-c920-4dc4-8525-53ba19eb3bc3 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:21.093577 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46093513-c920-4dc4-8525-53ba19eb3bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:21.094067 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46093513-c920-4dc4-8525-53ba19eb3bc3 None admin] PATH_INFO: `/v3/users/f8579d2130b94c37ad6f44b1a88786bf` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:21.122718 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46093513-c920-4dc4-8525-53ba19eb3bc3 None admin] RBAC: Authorizing `identity:get_user(user_id=f8579d2130b94c37ad6f44b1a88786bf)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:21.124579 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46093513-c920-4dc4-8525-53ba19eb3bc3 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:21.130208 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 58/114] 10.222.0.102 () {56 vars in 1182 bytes} [Thu Sep 1 17:17:20 2022] GET /identity/v3/users/f8579d2130b94c37ad6f44b1a88786bf => generated 290 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:21.136594 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46093513-c920-4dc4-8525-53ba19eb3bc3 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:21.217037 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e0131f-0ca5-4033-a95c-bcd3f30057b6 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:21.256365 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f29e3f-92ed-4037-bd23-f519de6cf579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:21.257049 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f29e3f-92ed-4037-bd23-f519de6cf579 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:21.257362 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f29e3f-92ed-4037-bd23-f519de6cf579 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:21.257643 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f29e3f-92ed-4037-bd23-f519de6cf579 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:21.268552 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f29e3f-92ed-4037-bd23-f519de6cf579 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:17:21.274010 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-67f29e3f-92ed-4037-bd23-f519de6cf579 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 17:17:21.274010 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:17:21.274010 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:17:21.274010 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:17:21.274010 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:17:21.274010 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:17:21.274010 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:17:21.274010 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:17:21.274010 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:17:21.274010 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:17:21.274010 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:17:21.274010 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:17:21.274010 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:17:21.274010 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:17:21.274010 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:17:21.274010 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:17:21.274857 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:17:21.274857 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:17:21.274857 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:17:21.274857 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:17:21.274857 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:17:21.274857 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:17:21.274857 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:17:21.274857 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:17:21.274857 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:17:21.274857 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:17:21.274857 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:17:21.274857 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:17:21.274857 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:17:21.274857 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:17:21.274857 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:17:21.274857 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:17:21.275592 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:17:21.275592 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:17:21.275592 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:17:21.275592 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:17:21.275592 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:17:21.275592 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:17:21.275592 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:17:21.275592 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:17:21.275592 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:17:21.275592 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:17:21.275592 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 17:17:21.275592 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:17:21.275592 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 57/115] 10.222.0.102 () {56 vars in 1119 bytes} [Thu Sep 1 17:17:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:17:21.276207 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f29e3f-92ed-4037-bd23-f519de6cf579 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:21.287691 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46093513-c920-4dc4-8525-53ba19eb3bc3 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:21.289900 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:21.338633 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963d4864-8378-4898-93d2-709e26b50ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:21.339709 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963d4864-8378-4898-93d2-709e26b50ed2 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:21.339995 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963d4864-8378-4898-93d2-709e26b50ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:21.340271 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963d4864-8378-4898-93d2-709e26b50ed2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:21.373767 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:21.405628 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f29e3f-92ed-4037-bd23-f519de6cf579 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:21.420561 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963d4864-8378-4898-93d2-709e26b50ed2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:21.422163 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963d4864-8378-4898-93d2-709e26b50ed2 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:21.442522 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af7a130-fc98-42f3-9eba-a3e5b9758623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:21.443157 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af7a130-fc98-42f3-9eba-a3e5b9758623 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:21.443439 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af7a130-fc98-42f3-9eba-a3e5b9758623 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:21.444030 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af7a130-fc98-42f3-9eba-a3e5b9758623 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:21.455903 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af7a130-fc98-42f3-9eba-a3e5b9758623 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:17:21.461748 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-6af7a130-fc98-42f3-9eba-a3e5b9758623 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:17:21.461748 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:17:21.461748 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:17:21.461748 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:17:21.461748 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:17:21.461748 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:17:21.461748 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:17:21.461748 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:17:21.461748 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:17:21.461748 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:17:21.461748 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:17:21.461748 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:17:21.461748 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:17:21.461748 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:17:21.461748 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:17:21.461748 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:17:21.462586 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:17:21.462586 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:17:21.462586 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:17:21.462586 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:17:21.462586 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:17:21.462586 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:17:21.462586 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:17:21.462586 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:17:21.462586 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:17:21.462586 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:17:21.462586 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:17:21.462586 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:17:21.462586 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:17:21.462586 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:17:21.462586 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:17:21.462586 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:17:21.463747 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:17:21.463747 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:17:21.463747 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:17:21.463747 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:17:21.463747 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:17:21.463747 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:17:21.463747 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:17:21.463747 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:17:21.463747 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:17:21.463747 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:17:21.463747 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:17:21.463747 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:17:21.463747 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 58/116] 10.222.0.102 () {56 vars in 1101 bytes} [Thu Sep 1 17:17:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:17:21.464561 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af7a130-fc98-42f3-9eba-a3e5b9758623 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:21.539380 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 59/117] 10.222.0.102 () {58 vars in 1287 bytes} [Thu Sep 1 17:17:21 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 17:17:21.540024 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:21.540263 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963d4864-8378-4898-93d2-709e26b50ed2 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:21.576367 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af7a130-fc98-42f3-9eba-a3e5b9758623 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:21.604172 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:21.614277 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95ed29c-deda-4ecf-bc8d-75947a8a2542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:21.614912 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95ed29c-deda-4ecf-bc8d-75947a8a2542 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:21.615191 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95ed29c-deda-4ecf-bc8d-75947a8a2542 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:21.615462 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95ed29c-deda-4ecf-bc8d-75947a8a2542 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:21.625936 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95ed29c-deda-4ecf-bc8d-75947a8a2542 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:17:21.630831 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-c95ed29c-deda-4ecf-bc8d-75947a8a2542 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:17:21.630831 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:17:21.630831 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:17:21.630831 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:17:21.630831 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:17:21.630831 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:17:21.630831 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:17:21.630831 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:17:21.630831 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:17:21.630831 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:17:21.630831 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:17:21.630831 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:17:21.630831 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:17:21.630831 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:17:21.630831 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:17:21.630831 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:17:21.632178 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:17:21.632178 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:17:21.632178 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:17:21.632178 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:17:21.632178 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:17:21.632178 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:17:21.632178 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:17:21.632178 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:17:21.632178 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:17:21.632178 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:17:21.632178 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:17:21.632178 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:17:21.632178 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:17:21.632178 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:17:21.632178 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:17:21.632178 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:17:21.632919 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:17:21.632919 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:17:21.632919 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:17:21.632919 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:17:21.632919 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:17:21.632919 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:17:21.632919 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:17:21.632919 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:17:21.632919 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:17:21.632919 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:17:21.632919 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:17:21.632919 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:17:21.632919 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 59/118] 10.222.0.102 () {56 vars in 1104 bytes} [Thu Sep 1 17:17:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:17:21.633663 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95ed29c-deda-4ecf-bc8d-75947a8a2542 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:21.655752 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963d4864-8378-4898-93d2-709e26b50ed2 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:21.694685 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003e0ef8-b7ea-44ba-a836-1f3c0eaa1d18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:21.695935 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003e0ef8-b7ea-44ba-a836-1f3c0eaa1d18 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:21.696413 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003e0ef8-b7ea-44ba-a836-1f3c0eaa1d18 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:21.696853 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003e0ef8-b7ea-44ba-a836-1f3c0eaa1d18 None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:21.697823 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003e0ef8-b7ea-44ba-a836-1f3c0eaa1d18 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:17:21.698438 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003e0ef8-b7ea-44ba-a836-1f3c0eaa1d18 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:21.700387 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003e0ef8-b7ea-44ba-a836-1f3c0eaa1d18 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:21.710400 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 60/119] 10.222.0.102 () {56 vars in 1116 bytes} [Thu Sep 1 17:17:21 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:21.711905 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003e0ef8-b7ea-44ba-a836-1f3c0eaa1d18 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:21.742591 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95ed29c-deda-4ecf-bc8d-75947a8a2542 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:21.771859 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:21.779842 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d218e9f-7fc9-43e2-8b76-9f04de5be985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:21.780563 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d218e9f-7fc9-43e2-8b76-9f04de5be985 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:21.780950 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d218e9f-7fc9-43e2-8b76-9f04de5be985 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:21.781294 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d218e9f-7fc9-43e2-8b76-9f04de5be985 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:21.792596 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d218e9f-7fc9-43e2-8b76-9f04de5be985 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:17:21.797610 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-4d218e9f-7fc9-43e2-8b76-9f04de5be985 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:17:21.797610 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:17:21.797610 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:17:21.797610 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:17:21.797610 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:17:21.797610 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:17:21.797610 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:17:21.797610 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:17:21.797610 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:17:21.797610 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:17:21.797610 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:17:21.797610 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:17:21.797610 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:17:21.797610 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:17:21.797610 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:17:21.797610 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:17:21.798579 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:17:21.798579 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:17:21.798579 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:17:21.798579 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:17:21.798579 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:17:21.798579 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:17:21.798579 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:17:21.798579 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:17:21.798579 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:17:21.798579 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:17:21.798579 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:17:21.798579 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:17:21.798579 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:17:21.798579 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:17:21.798579 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:17:21.798579 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:17:21.799598 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:17:21.799598 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:17:21.799598 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:17:21.799598 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:17:21.799598 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:17:21.799598 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:17:21.799598 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:17:21.799598 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:17:21.799598 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:17:21.799598 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:17:21.799598 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:17:21.799598 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:17:21.801484 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 60/120] 10.222.0.102 () {56 vars in 1104 bytes} [Thu Sep 1 17:17:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:17:21.802383 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d218e9f-7fc9-43e2-8b76-9f04de5be985 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:21.823847 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003e0ef8-b7ea-44ba-a836-1f3c0eaa1d18 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:21.852456 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:21.869112 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392f0cc2-a347-473f-ba27-9178dc5519bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:21.869986 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392f0cc2-a347-473f-ba27-9178dc5519bc None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:21.870271 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392f0cc2-a347-473f-ba27-9178dc5519bc None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:21.870975 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392f0cc2-a347-473f-ba27-9178dc5519bc None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:21.871942 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392f0cc2-a347-473f-ba27-9178dc5519bc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:17:21.872251 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392f0cc2-a347-473f-ba27-9178dc5519bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:21.873995 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392f0cc2-a347-473f-ba27-9178dc5519bc None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:21.890936 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 61/121] 10.222.0.102 () {56 vars in 1104 bytes} [Thu Sep 1 17:17:21 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:21.890936 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392f0cc2-a347-473f-ba27-9178dc5519bc None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:21.909374 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d218e9f-7fc9-43e2-8b76-9f04de5be985 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:21.947201 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aecbea6-30cf-43d2-bd8c-8e78f7eae21a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:21.947829 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aecbea6-30cf-43d2-bd8c-8e78f7eae21a None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:21.948107 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aecbea6-30cf-43d2-bd8c-8e78f7eae21a None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:21.948375 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aecbea6-30cf-43d2-bd8c-8e78f7eae21a None admin] PATH_INFO: `/v3/projects/8e88c67b942f45108c95a833547f9bc8` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:21.949639 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aecbea6-30cf-43d2-bd8c-8e78f7eae21a None admin] RBAC: Authorizing `identity:get_project(project_id=8e88c67b942f45108c95a833547f9bc8)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:21.951080 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aecbea6-30cf-43d2-bd8c-8e78f7eae21a None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:21.952694 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 61/122] 10.222.0.102 () {56 vars in 1191 bytes} [Thu Sep 1 17:17:21 2022] GET /identity/v3/projects/8e88c67b942f45108c95a833547f9bc8 => generated 299 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:21.953489 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aecbea6-30cf-43d2-bd8c-8e78f7eae21a None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:22.003457 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392f0cc2-a347-473f-ba27-9178dc5519bc None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:22.017992 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:22.043772 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae46255-88f2-4244-905e-04c9329fbd35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:22.044394 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae46255-88f2-4244-905e-04c9329fbd35 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:22.044673 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae46255-88f2-4244-905e-04c9329fbd35 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:22.044943 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae46255-88f2-4244-905e-04c9329fbd35 None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:22.045660 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae46255-88f2-4244-905e-04c9329fbd35 None admin] RBAC: Adding query filter params (name=member) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:17:22.045970 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae46255-88f2-4244-905e-04c9329fbd35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:22.047447 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae46255-88f2-4244-905e-04c9329fbd35 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:22.057628 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 62/123] 10.222.0.102 () {56 vars in 1106 bytes} [Thu Sep 1 17:17:21 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:22.058671 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae46255-88f2-4244-905e-04c9329fbd35 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:22.065094 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aecbea6-30cf-43d2-bd8c-8e78f7eae21a None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:22.100385 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8ea9b6-51cf-46f3-8bef-c61b32870ca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:22.101040 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8ea9b6-51cf-46f3-8bef-c61b32870ca4 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:22.101622 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8ea9b6-51cf-46f3-8bef-c61b32870ca4 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:22.101946 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8ea9b6-51cf-46f3-8bef-c61b32870ca4 None admin] PATH_INFO: `/v3/users/27f309e66b2c4003a820573ada1b0958` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:22.103082 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8ea9b6-51cf-46f3-8bef-c61b32870ca4 None admin] RBAC: Authorizing `identity:get_user(user_id=27f309e66b2c4003a820573ada1b0958)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:22.104531 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8ea9b6-51cf-46f3-8bef-c61b32870ca4 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:22.106344 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 62/124] 10.222.0.102 () {56 vars in 1182 bytes} [Thu Sep 1 17:17:21 2022] GET /identity/v3/users/27f309e66b2c4003a820573ada1b0958 => generated 276 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:22.106890 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8ea9b6-51cf-46f3-8bef-c61b32870ca4 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:22.111489 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:22.171458 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae46255-88f2-4244-905e-04c9329fbd35 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:22.207074 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8915fb6d-15ba-45ff-9d48-655d2fcea429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:22.207695 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8915fb6d-15ba-45ff-9d48-655d2fcea429 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:22.207978 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8915fb6d-15ba-45ff-9d48-655d2fcea429 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:22.208281 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8915fb6d-15ba-45ff-9d48-655d2fcea429 None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:22.209162 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8915fb6d-15ba-45ff-9d48-655d2fcea429 None admin] RBAC: Adding query filter params (name=member) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:17:22.209263 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8915fb6d-15ba-45ff-9d48-655d2fcea429 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:22.210590 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8915fb6d-15ba-45ff-9d48-655d2fcea429 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:22.218064 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8ea9b6-51cf-46f3-8bef-c61b32870ca4 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:22.218854 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 63/125] 10.222.0.102 () {56 vars in 1106 bytes} [Thu Sep 1 17:17:22 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:22.219646 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8915fb6d-15ba-45ff-9d48-655d2fcea429 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:22.255532 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:22.265004 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6236195e-bcdc-4f06-89b4-5d680bc0e172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:22.265660 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6236195e-bcdc-4f06-89b4-5d680bc0e172 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:22.265940 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6236195e-bcdc-4f06-89b4-5d680bc0e172 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:22.266214 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6236195e-bcdc-4f06-89b4-5d680bc0e172 None admin] PATH_INFO: `/v3/users/e6c9197e0f844432a03b9e75fc87712b` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:22.267332 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6236195e-bcdc-4f06-89b4-5d680bc0e172 None admin] RBAC: Authorizing `identity:get_user(user_id=e6c9197e0f844432a03b9e75fc87712b)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:22.268778 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6236195e-bcdc-4f06-89b4-5d680bc0e172 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:22.274996 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 63/126] 10.222.0.102 () {56 vars in 1182 bytes} [Thu Sep 1 17:17:22 2022] GET /identity/v3/users/e6c9197e0f844432a03b9e75fc87712b => generated 248 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:22.274996 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6236195e-bcdc-4f06-89b4-5d680bc0e172 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:22.329297 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8915fb6d-15ba-45ff-9d48-655d2fcea429 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:22.338410 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:22.367585 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f2770d-3782-4d8a-9a44-9a0fe0a93ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:22.368213 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f2770d-3782-4d8a-9a44-9a0fe0a93ef3 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:22.368491 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f2770d-3782-4d8a-9a44-9a0fe0a93ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:22.368766 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f2770d-3782-4d8a-9a44-9a0fe0a93ef3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:22.369523 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f2770d-3782-4d8a-9a44-9a0fe0a93ef3 None admin] RBAC: Adding query filter params (role.id=ab816d03be024513884447335f2de35a, scope.project.id=8e88c67b942f45108c95a833547f9bc8, user.id=f8579d2130b94c37ad6f44b1a88786bf) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:17:22.369840 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f2770d-3782-4d8a-9a44-9a0fe0a93ef3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:22.371297 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f2770d-3782-4d8a-9a44-9a0fe0a93ef3 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:22.377003 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 64/127] 10.222.0.102 () {56 vars in 1379 bytes} [Thu Sep 1 17:17:22 2022] GET /identity/v3/role_assignments?user.id=f8579d2130b94c37ad6f44b1a88786bf&scope.project.id=8e88c67b942f45108c95a833547f9bc8&role.id=ab816d03be024513884447335f2de35a => generated 260 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:22.385513 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f2770d-3782-4d8a-9a44-9a0fe0a93ef3 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:22.399111 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6236195e-bcdc-4f06-89b4-5d680bc0e172 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:22.450515 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29b3644-192d-4107-8cff-f3651a11e8b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:22.451192 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29b3644-192d-4107-8cff-f3651a11e8b9 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:22.451576 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29b3644-192d-4107-8cff-f3651a11e8b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:22.451970 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29b3644-192d-4107-8cff-f3651a11e8b9 None admin] PATH_INFO: `/v3/users/27f309e66b2c4003a820573ada1b0958` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:22.453432 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29b3644-192d-4107-8cff-f3651a11e8b9 None admin] RBAC: Authorizing `identity:get_user(user_id=27f309e66b2c4003a820573ada1b0958)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:22.455110 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29b3644-192d-4107-8cff-f3651a11e8b9 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:22.457052 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 64/128] 10.222.0.102 () {56 vars in 1182 bytes} [Thu Sep 1 17:17:22 2022] GET /identity/v3/users/27f309e66b2c4003a820573ada1b0958 => generated 276 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:22.458104 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29b3644-192d-4107-8cff-f3651a11e8b9 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:22.506618 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f2770d-3782-4d8a-9a44-9a0fe0a93ef3 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:22.529700 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:22.548455 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e598825-2f75-425f-b288-7de7b4670ac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:22.549416 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e598825-2f75-425f-b288-7de7b4670ac8 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:22.549854 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e598825-2f75-425f-b288-7de7b4670ac8 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:22.550136 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e598825-2f75-425f-b288-7de7b4670ac8 None admin] PATH_INFO: `/v3/projects/8e88c67b942f45108c95a833547f9bc8` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:22.551959 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e598825-2f75-425f-b288-7de7b4670ac8 None admin] RBAC: Authorizing `identity:get_project(project_id=8e88c67b942f45108c95a833547f9bc8)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:22.553443 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e598825-2f75-425f-b288-7de7b4670ac8 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:22.555682 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 65/129] 10.222.0.102 () {56 vars in 1191 bytes} [Thu Sep 1 17:17:22 2022] GET /identity/v3/projects/8e88c67b942f45108c95a833547f9bc8 => generated 299 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:22.556360 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e598825-2f75-425f-b288-7de7b4670ac8 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:22.575326 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29b3644-192d-4107-8cff-f3651a11e8b9 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:22.610220 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:22.620775 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09490b34-6813-432d-ac56-f41bdfd286a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:22.621446 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09490b34-6813-432d-ac56-f41bdfd286a5 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:22.621730 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09490b34-6813-432d-ac56-f41bdfd286a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:22.622006 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09490b34-6813-432d-ac56-f41bdfd286a5 None admin] PATH_INFO: `/v3/users/27f309e66b2c4003a820573ada1b0958` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:22.623697 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09490b34-6813-432d-ac56-f41bdfd286a5 None admin] RBAC: Authorizing `identity:get_user(user_id=27f309e66b2c4003a820573ada1b0958)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:22.625457 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09490b34-6813-432d-ac56-f41bdfd286a5 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:22.628183 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 65/130] 10.222.0.102 () {56 vars in 1182 bytes} [Thu Sep 1 17:17:22 2022] GET /identity/v3/users/27f309e66b2c4003a820573ada1b0958 => generated 276 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:22.628852 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09490b34-6813-432d-ac56-f41bdfd286a5 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:22.682403 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e598825-2f75-425f-b288-7de7b4670ac8 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:22.724902 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b77e2a-ce3d-48ca-b3f0-5e05bb7552f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:22.725531 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b77e2a-ce3d-48ca-b3f0-5e05bb7552f5 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:22.725823 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b77e2a-ce3d-48ca-b3f0-5e05bb7552f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:22.726187 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b77e2a-ce3d-48ca-b3f0-5e05bb7552f5 None admin] PATH_INFO: `/v3/projects/8e88c67b942f45108c95a833547f9bc8` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:22.727591 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b77e2a-ce3d-48ca-b3f0-5e05bb7552f5 None admin] RBAC: Authorizing `identity:get_project(project_id=8e88c67b942f45108c95a833547f9bc8)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:22.729079 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b77e2a-ce3d-48ca-b3f0-5e05bb7552f5 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:22.731142 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 66/131] 10.222.0.102 () {56 vars in 1191 bytes} [Thu Sep 1 17:17:22 2022] GET /identity/v3/projects/8e88c67b942f45108c95a833547f9bc8 => generated 299 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:22.732162 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b77e2a-ce3d-48ca-b3f0-5e05bb7552f5 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:22.766233 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09490b34-6813-432d-ac56-f41bdfd286a5 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:22.804842 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:22.812227 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40675f8-0086-44a8-8026-ebff548f6172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:22.813165 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40675f8-0086-44a8-8026-ebff548f6172 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:22.813488 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40675f8-0086-44a8-8026-ebff548f6172 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:22.813768 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40675f8-0086-44a8-8026-ebff548f6172 None admin] PATH_INFO: `/v3/projects/dc1ae957803645b4abe7503e4399bf10` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:22.815533 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40675f8-0086-44a8-8026-ebff548f6172 None admin] RBAC: Authorizing `identity:get_project(project_id=dc1ae957803645b4abe7503e4399bf10)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:22.817762 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40675f8-0086-44a8-8026-ebff548f6172 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:22.822265 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 66/132] 10.222.0.102 () {56 vars in 1191 bytes} [Thu Sep 1 17:17:22 2022] GET /identity/v3/projects/dc1ae957803645b4abe7503e4399bf10 => generated 313 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:22.823619 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40675f8-0086-44a8-8026-ebff548f6172 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:22.853257 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b77e2a-ce3d-48ca-b3f0-5e05bb7552f5 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:22.866645 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:22.901531 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a669fad5-f1e0-40cf-9734-8955e40caf3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:22.902667 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a669fad5-f1e0-40cf-9734-8955e40caf3d None admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:22.903306 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a669fad5-f1e0-40cf-9734-8955e40caf3d None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:22.903777 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a669fad5-f1e0-40cf-9734-8955e40caf3d None admin] PATH_INFO: `/v3/projects/8e88c67b942f45108c95a833547f9bc8/users/27f309e66b2c4003a820573ada1b0958/roles/abef78c185c74c16a2c7e8a170583f9f` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:22.906298 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a669fad5-f1e0-40cf-9734-8955e40caf3d None admin] RBAC: Authorizing `identity:create_grant(project_id=8e88c67b942f45108c95a833547f9bc8, user_id=27f309e66b2c4003a820573ada1b0958, role_id=abef78c185c74c16a2c7e8a170583f9f)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:22.908009 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a669fad5-f1e0-40cf-9734-8955e40caf3d None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:22.924304 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 67/133] 10.222.0.102 () {58 vars in 1444 bytes} [Thu Sep 1 17:17:22 2022] PUT /identity/v3/projects/8e88c67b942f45108c95a833547f9bc8/users/27f309e66b2c4003a820573ada1b0958/roles/abef78c185c74c16a2c7e8a170583f9f => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:17:22.925890 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a669fad5-f1e0-40cf-9734-8955e40caf3d None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:22.961147 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40675f8-0086-44a8-8026-ebff548f6172 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:23.022662 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf43e9d6-d9c3-492a-84e0-1ad6e5431cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:23.023665 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf43e9d6-d9c3-492a-84e0-1ad6e5431cff None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:23.024151 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf43e9d6-d9c3-492a-84e0-1ad6e5431cff None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:23.024625 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf43e9d6-d9c3-492a-84e0-1ad6e5431cff None admin] PATH_INFO: `/v3/projects/8e88c67b942f45108c95a833547f9bc8` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:23.026830 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf43e9d6-d9c3-492a-84e0-1ad6e5431cff None admin] RBAC: Authorizing `identity:get_project(project_id=8e88c67b942f45108c95a833547f9bc8)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:23.029414 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf43e9d6-d9c3-492a-84e0-1ad6e5431cff None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:23.031925 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 67/134] 10.222.0.102 () {56 vars in 1191 bytes} [Thu Sep 1 17:17:22 2022] GET /identity/v3/projects/8e88c67b942f45108c95a833547f9bc8 => generated 299 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:23.033611 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf43e9d6-d9c3-492a-84e0-1ad6e5431cff None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:23.057463 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a669fad5-f1e0-40cf-9734-8955e40caf3d None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:23.109066 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8625acef-7c2f-4229-bef6-a33133cee6d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:23.109709 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8625acef-7c2f-4229-bef6-a33133cee6d5 None admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:23.109993 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8625acef-7c2f-4229-bef6-a33133cee6d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:23.110289 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8625acef-7c2f-4229-bef6-a33133cee6d5 None admin] PATH_INFO: `/v3/projects/8e88c67b942f45108c95a833547f9bc8/users/e6c9197e0f844432a03b9e75fc87712b/roles/36577a6f46d642d4b81805ede9872490` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:23.113258 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8625acef-7c2f-4229-bef6-a33133cee6d5 None admin] RBAC: Authorizing `identity:create_grant(project_id=8e88c67b942f45108c95a833547f9bc8, user_id=e6c9197e0f844432a03b9e75fc87712b, role_id=36577a6f46d642d4b81805ede9872490)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:23.114817 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8625acef-7c2f-4229-bef6-a33133cee6d5 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:23.129843 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:23.132271 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 68/135] 10.222.0.102 () {58 vars in 1444 bytes} [Thu Sep 1 17:17:22 2022] PUT /identity/v3/projects/8e88c67b942f45108c95a833547f9bc8/users/e6c9197e0f844432a03b9e75fc87712b/roles/36577a6f46d642d4b81805ede9872490 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:17:23.133680 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650c55bb-7244-4e63-93d6-6496b402909a None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:23.133799 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650c55bb-7244-4e63-93d6-6496b402909a None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:23.134003 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650c55bb-7244-4e63-93d6-6496b402909a None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:23.135335 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 69/136] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:17:23 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:17:23.136073 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650c55bb-7244-4e63-93d6-6496b402909a None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:23.180921 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf43e9d6-d9c3-492a-84e0-1ad6e5431cff None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:23.183767 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:23.261647 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4851b9f-a426-489c-8594-e9cdc9a72af3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:23.262811 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4851b9f-a426-489c-8594-e9cdc9a72af3 None admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:23.263292 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4851b9f-a426-489c-8594-e9cdc9a72af3 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:23.263770 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4851b9f-a426-489c-8594-e9cdc9a72af3 None admin] PATH_INFO: `/v3/projects/dc1ae957803645b4abe7503e4399bf10/users/27f309e66b2c4003a820573ada1b0958/roles/b75cb360925e4ba5a65f14965c103ffd` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:23.266692 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4851b9f-a426-489c-8594-e9cdc9a72af3 None admin] RBAC: Authorizing `identity:create_grant(project_id=dc1ae957803645b4abe7503e4399bf10, user_id=27f309e66b2c4003a820573ada1b0958, role_id=b75cb360925e4ba5a65f14965c103ffd)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:23.269666 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4851b9f-a426-489c-8594-e9cdc9a72af3 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:23.296081 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 68/137] 10.222.0.102 () {58 vars in 1444 bytes} [Thu Sep 1 17:17:23 2022] PUT /identity/v3/projects/dc1ae957803645b4abe7503e4399bf10/users/27f309e66b2c4003a820573ada1b0958/roles/b75cb360925e4ba5a65f14965c103ffd => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:17:23.297087 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d1eba7-7675-4862-8466-9d983a79d19d None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:23.297947 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d1eba7-7675-4862-8466-9d983a79d19d None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:23.298251 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d1eba7-7675-4862-8466-9d983a79d19d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:23.323968 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-f5d1eba7-7675-4862-8466-9d983a79d19d None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:17:23.326057 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-f5d1eba7-7675-4862-8466-9d983a79d19d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:17:23.335834 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650c55bb-7244-4e63-93d6-6496b402909a None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:23.380493 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629651f5-7c6e-4273-878b-abe1ee8b64f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:23.382104 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629651f5-7c6e-4273-878b-abe1ee8b64f2 None admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:23.382622 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629651f5-7c6e-4273-878b-abe1ee8b64f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:23.383154 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629651f5-7c6e-4273-878b-abe1ee8b64f2 None admin] PATH_INFO: `/v3/projects/8e88c67b942f45108c95a833547f9bc8/users/27f309e66b2c4003a820573ada1b0958/roles/b75cb360925e4ba5a65f14965c103ffd` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:23.386554 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629651f5-7c6e-4273-878b-abe1ee8b64f2 None admin] RBAC: Authorizing `identity:create_grant(project_id=8e88c67b942f45108c95a833547f9bc8, user_id=27f309e66b2c4003a820573ada1b0958, role_id=b75cb360925e4ba5a65f14965c103ffd)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:23.388293 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629651f5-7c6e-4273-878b-abe1ee8b64f2 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:23.424134 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 69/138] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:17:23 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:17:23.427143 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 70/139] 10.222.0.102 () {58 vars in 1444 bytes} [Thu Sep 1 17:17:23 2022] PUT /identity/v3/projects/8e88c67b942f45108c95a833547f9bc8/users/27f309e66b2c4003a820573ada1b0958/roles/b75cb360925e4ba5a65f14965c103ffd => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:17:23.814554 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d1eba7-7675-4862-8466-9d983a79d19d None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:23.889805 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:23.942043 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d1eba7-7675-4862-8466-9d983a79d19d None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:23.978973 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1f80c9-99aa-4d0b-8dbd-15d53c1bcdec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:23.979608 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1f80c9-99aa-4d0b-8dbd-15d53c1bcdec None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:23.979957 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1f80c9-99aa-4d0b-8dbd-15d53c1bcdec None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:23.980238 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1f80c9-99aa-4d0b-8dbd-15d53c1bcdec None admin] PATH_INFO: `/v3/domains/default` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:23.981387 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1f80c9-99aa-4d0b-8dbd-15d53c1bcdec None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:23.982826 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1f80c9-99aa-4d0b-8dbd-15d53c1bcdec None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:23.984613 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 70/140] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:17:23 2022] GET /identity/v3/domains/default => generated 200 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:23.990054 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629651f5-7c6e-4273-878b-abe1ee8b64f2 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:24.085940 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:24.101164 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629651f5-7c6e-4273-878b-abe1ee8b64f2 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:24.137664 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333b7b80-2cc1-445e-b434-42364074c29a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:24.138334 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333b7b80-2cc1-445e-b434-42364074c29a None admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:24.138628 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333b7b80-2cc1-445e-b434-42364074c29a None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:24.138911 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333b7b80-2cc1-445e-b434-42364074c29a None admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:24.139588 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333b7b80-2cc1-445e-b434-42364074c29a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:24.141044 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333b7b80-2cc1-445e-b434-42364074c29a None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:24.167235 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 71/141] 10.222.0.102 () {60 vars in 1137 bytes} [Thu Sep 1 17:17:23 2022] POST /identity/v3/users => generated 284 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:17:25.255062 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10017414-11e7-4c40-952f-fcd4e1740881 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:25.255538 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10017414-11e7-4c40-952f-fcd4e1740881 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:25.255538 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10017414-11e7-4c40-952f-fcd4e1740881 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:25.257219 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 71/142] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:17:25 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:17:25.262967 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26a1a5e-fb9c-4c2d-9989-239c0be8ac51 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:25.263432 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26a1a5e-fb9c-4c2d-9989-239c0be8ac51 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:25.263614 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26a1a5e-fb9c-4c2d-9989-239c0be8ac51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:25.284771 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-a26a1a5e-fb9c-4c2d-9989-239c0be8ac51 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:17:25.286120 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-a26a1a5e-fb9c-4c2d-9989-239c0be8ac51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:17:25.362253 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 72/143] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:17:25 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 17:17:25.699527 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10017414-11e7-4c40-952f-fcd4e1740881 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:25.801894 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b84a3e-60fb-4b9c-82a8-b8751588e1f7 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:25.802235 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b84a3e-60fb-4b9c-82a8-b8751588e1f7 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:25.802516 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b84a3e-60fb-4b9c-82a8-b8751588e1f7 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:25.803881 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 73/144] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:17:25 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:17:25.820421 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa93b7d-dd5c-4efa-8bc3-1a53f82b2389 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:25.820709 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa93b7d-dd5c-4efa-8bc3-1a53f82b2389 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:25.820990 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa93b7d-dd5c-4efa-8bc3-1a53f82b2389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:25.851902 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-eaa93b7d-dd5c-4efa-8bc3-1a53f82b2389 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:17:25.851902 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-eaa93b7d-dd5c-4efa-8bc3-1a53f82b2389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:17:25.934082 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10017414-11e7-4c40-952f-fcd4e1740881 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:25.967151 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 74/145] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:17:25 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 17:17:26.009672 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:26.038572 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8360af52-3692-494b-88c8-cfccc90b4ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:26.039312 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8360af52-3692-494b-88c8-cfccc90b4ca1 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:26.039760 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8360af52-3692-494b-88c8-cfccc90b4ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:26.040158 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8360af52-3692-494b-88c8-cfccc90b4ca1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:26.045262 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6760c16f-4b20-4b8e-9072-78dfba022ed4 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:26.045612 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6760c16f-4b20-4b8e-9072-78dfba022ed4 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:26.046267 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6760c16f-4b20-4b8e-9072-78dfba022ed4 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:26.047649 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 75/146] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:17:26 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:17:26.058553 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8360af52-3692-494b-88c8-cfccc90b4ca1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:17:26.068870 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-8360af52-3692-494b-88c8-cfccc90b4ca1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 17:17:26.068870 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:17:26.068870 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:17:26.068870 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:17:26.068870 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:17:26.068870 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:17:26.068870 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:17:26.068870 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:17:26.068870 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:17:26.068870 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:17:26.068870 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:17:26.068870 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:17:26.068870 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:17:26.068870 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:17:26.068870 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:17:26.068870 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:17:26.069827 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:17:26.069827 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:17:26.069827 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:17:26.069827 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:17:26.069827 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:17:26.069827 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:17:26.069827 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:17:26.069827 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:17:26.069827 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:17:26.069827 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:17:26.069827 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:17:26.069827 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:17:26.069827 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:17:26.069827 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:17:26.069827 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:17:26.069827 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:17:26.070554 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:17:26.070554 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:17:26.070554 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:17:26.070554 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:17:26.070554 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:17:26.070554 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:17:26.070554 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:17:26.070554 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:17:26.070554 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:17:26.070554 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:17:26.070554 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 17:17:26.070554 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:17:26.070554 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 72/147] 10.222.0.102 () {56 vars in 1104 bytes} [Thu Sep 1 17:17:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 371 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:17:26.074514 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198fba9d-786d-4b01-9435-371ddd0979a8 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:26.074803 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198fba9d-786d-4b01-9435-371ddd0979a8 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:26.075080 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198fba9d-786d-4b01-9435-371ddd0979a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:26.076186 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8360af52-3692-494b-88c8-cfccc90b4ca1 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:26.093018 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:26.114714 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-198fba9d-786d-4b01-9435-371ddd0979a8 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:17:26.116004 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-198fba9d-786d-4b01-9435-371ddd0979a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:17:26.206506 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 76/148] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:17:26 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:17:26.208143 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5f0ac9-c04b-4073-90a6-871b32d74365 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:26.208435 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5f0ac9-c04b-4073-90a6-871b32d74365 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:26.209620 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5f0ac9-c04b-4073-90a6-871b32d74365 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:26.210932 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 77/149] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:17:26 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:17:26.216674 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167cc4b5-f00a-43e5-80df-5326dd6d9eed None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:26.216965 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167cc4b5-f00a-43e5-80df-5326dd6d9eed None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:26.217720 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167cc4b5-f00a-43e5-80df-5326dd6d9eed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:26.243868 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-167cc4b5-f00a-43e5-80df-5326dd6d9eed None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:17:26.245090 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-167cc4b5-f00a-43e5-80df-5326dd6d9eed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:17:26.246813 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8360af52-3692-494b-88c8-cfccc90b4ca1 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:26.322040 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 78/150] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:17:26 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 17:17:26.355351 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93efd69d-9e12-40f5-8465-1f2d2cedeeda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:26.356436 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93efd69d-9e12-40f5-8465-1f2d2cedeeda None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:26.356791 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93efd69d-9e12-40f5-8465-1f2d2cedeeda None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:26.357141 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93efd69d-9e12-40f5-8465-1f2d2cedeeda None admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:26.357951 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93efd69d-9e12-40f5-8465-1f2d2cedeeda None admin] RBAC: Adding query filter params (name=reader) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:17:26.358332 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93efd69d-9e12-40f5-8465-1f2d2cedeeda None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:26.359916 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93efd69d-9e12-40f5-8465-1f2d2cedeeda None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:26.373941 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 73/151] 10.222.0.102 () {56 vars in 1106 bytes} [Thu Sep 1 17:17:26 2022] GET /identity/v3/roles?name=reader => generated 343 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:17:26.379989 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167cc4b5-f00a-43e5-80df-5326dd6d9eed None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:26.464758 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:26.495800 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9af8a7-f3c6-4370-ac70-70a4a913cc82 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:26.496394 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9af8a7-f3c6-4370-ac70-70a4a913cc82 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:26.496854 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9af8a7-f3c6-4370-ac70-70a4a913cc82 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:26.498487 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 74/152] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:17:26 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Sep 01 17:17:26.503867 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d6e0ce-187c-4988-a838-5843de6df889 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:26.504988 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d6e0ce-187c-4988-a838-5843de6df889 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:26.505622 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d6e0ce-187c-4988-a838-5843de6df889 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:26.513486 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167cc4b5-f00a-43e5-80df-5326dd6d9eed None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:26.550282 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-81d6e0ce-187c-4988-a838-5843de6df889 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:17:26.552035 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1c0083-604b-4653-9b97-259d96fcb03d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:26.552243 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-81d6e0ce-187c-4988-a838-5843de6df889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:17:26.552686 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1c0083-604b-4653-9b97-259d96fcb03d None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:26.552971 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1c0083-604b-4653-9b97-259d96fcb03d None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:26.553247 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1c0083-604b-4653-9b97-259d96fcb03d None admin] PATH_INFO: `/v3/users/f8579d2130b94c37ad6f44b1a88786bf` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:26.554431 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1c0083-604b-4653-9b97-259d96fcb03d None admin] RBAC: Authorizing `identity:get_user(user_id=f8579d2130b94c37ad6f44b1a88786bf)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:26.561332 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1c0083-604b-4653-9b97-259d96fcb03d None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:26.561332 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 79/153] 10.222.0.102 () {56 vars in 1182 bytes} [Thu Sep 1 17:17:26 2022] GET /identity/v3/users/f8579d2130b94c37ad6f44b1a88786bf => generated 290 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:26.564500 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1c0083-604b-4653-9b97-259d96fcb03d None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:26.658224 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:26.717254 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1c0083-604b-4653-9b97-259d96fcb03d None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:26.735600 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 75/154] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:17:26 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 17:17:26.736811 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d6e0ce-187c-4988-a838-5843de6df889 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:26.760826 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989cd8dc-f8cb-44f9-a9a4-544fbf1bb5d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:26.760826 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989cd8dc-f8cb-44f9-a9a4-544fbf1bb5d0 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:26.760826 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989cd8dc-f8cb-44f9-a9a4-544fbf1bb5d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:26.760826 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989cd8dc-f8cb-44f9-a9a4-544fbf1bb5d0 None admin] PATH_INFO: `/v3/projects/8e88c67b942f45108c95a833547f9bc8` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:26.761154 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989cd8dc-f8cb-44f9-a9a4-544fbf1bb5d0 None admin] RBAC: Authorizing `identity:get_project(project_id=8e88c67b942f45108c95a833547f9bc8)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:26.761154 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989cd8dc-f8cb-44f9-a9a4-544fbf1bb5d0 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:26.761154 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 80/155] 10.222.0.102 () {56 vars in 1191 bytes} [Thu Sep 1 17:17:26 2022] GET /identity/v3/projects/8e88c67b942f45108c95a833547f9bc8 => generated 299 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:26.762305 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989cd8dc-f8cb-44f9-a9a4-544fbf1bb5d0 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:26.773362 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:26.893697 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d6e0ce-187c-4988-a838-5843de6df889 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:26.932416 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989cd8dc-f8cb-44f9-a9a4-544fbf1bb5d0 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:27.002019 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3db1cc-ea49-4601-b746-f320f66939fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:27.002871 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3db1cc-ea49-4601-b746-f320f66939fd None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:27.003169 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3db1cc-ea49-4601-b746-f320f66939fd None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:27.003449 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3db1cc-ea49-4601-b746-f320f66939fd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:27.014185 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4213245e-1a03-4988-abe4-5686ec980bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:27.014761 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4213245e-1a03-4988-abe4-5686ec980bb9 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:27.015048 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4213245e-1a03-4988-abe4-5686ec980bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:27.015319 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4213245e-1a03-4988-abe4-5686ec980bb9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:27.018944 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3db1cc-ea49-4601-b746-f320f66939fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:17:27.024423 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-bd3db1cc-ea49-4601-b746-f320f66939fd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 17:17:27.024423 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:17:27.024423 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:17:27.024423 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:17:27.024423 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:17:27.024423 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:17:27.024423 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:17:27.024423 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:17:27.024423 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:17:27.024423 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:17:27.024423 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:17:27.024423 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:17:27.024423 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:17:27.024423 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:17:27.024423 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:17:27.024423 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:17:27.025288 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:17:27.025288 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:17:27.025288 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:17:27.025288 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:17:27.025288 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:17:27.025288 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:17:27.025288 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:17:27.025288 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:17:27.025288 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:17:27.025288 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:17:27.025288 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:17:27.025288 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:17:27.025288 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:17:27.025288 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:17:27.025288 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:17:27.025288 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:17:27.026055 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:17:27.026055 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:17:27.026055 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:17:27.026055 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:17:27.026055 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:17:27.026055 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:17:27.026055 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:17:27.026055 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:17:27.026055 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:17:27.026055 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:17:27.026055 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 17:17:27.026055 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:17:27.026055 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 81/156] 10.222.0.102 () {56 vars in 1119 bytes} [Thu Sep 1 17:17:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:17:27.027185 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3db1cc-ea49-4601-b746-f320f66939fd None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:27.049946 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4213245e-1a03-4988-abe4-5686ec980bb9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:17:27.055407 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-4213245e-1a03-4988-abe4-5686ec980bb9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:17:27.055407 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:17:27.055407 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:17:27.055407 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:17:27.055407 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:17:27.055407 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:17:27.055407 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:17:27.055407 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:17:27.055407 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:17:27.055407 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:17:27.055407 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:17:27.055407 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:17:27.055407 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:17:27.055407 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:17:27.055407 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:17:27.055407 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:17:27.056247 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:17:27.056247 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:17:27.056247 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:17:27.056247 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:17:27.056247 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:17:27.056247 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:17:27.056247 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:17:27.056247 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:17:27.056247 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:17:27.056247 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:17:27.056247 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:17:27.056247 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:17:27.056247 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:17:27.056247 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:17:27.056247 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:17:27.056247 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:17:27.058600 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:17:27.058600 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:17:27.058600 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:17:27.058600 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:17:27.058600 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:17:27.058600 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:17:27.058600 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:17:27.058600 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:17:27.058600 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:17:27.058600 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:17:27.058600 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:17:27.058600 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:17:27.058600 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 76/157] 10.222.0.102 () {56 vars in 1104 bytes} [Thu Sep 1 17:17:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:17:27.058600 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4213245e-1a03-4988-abe4-5686ec980bb9 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:27.097739 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:27.133651 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3db1cc-ea49-4601-b746-f320f66939fd None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:27.169342 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f2ce48-72d2-40e3-b4a7-1f958417a065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:27.170021 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f2ce48-72d2-40e3-b4a7-1f958417a065 None admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:27.170309 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f2ce48-72d2-40e3-b4a7-1f958417a065 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:27.170590 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f2ce48-72d2-40e3-b4a7-1f958417a065 None admin] PATH_INFO: `/v3/projects/8e88c67b942f45108c95a833547f9bc8/users/f8579d2130b94c37ad6f44b1a88786bf/roles/ab816d03be024513884447335f2de35a` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:27.172561 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f2ce48-72d2-40e3-b4a7-1f958417a065 None admin] RBAC: Authorizing `identity:create_grant(project_id=8e88c67b942f45108c95a833547f9bc8, user_id=f8579d2130b94c37ad6f44b1a88786bf, role_id=ab816d03be024513884447335f2de35a)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:27.174244 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f2ce48-72d2-40e3-b4a7-1f958417a065 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:27.189717 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 82/158] 10.222.0.102 () {58 vars in 1444 bytes} [Thu Sep 1 17:17:27 2022] PUT /identity/v3/projects/8e88c67b942f45108c95a833547f9bc8/users/f8579d2130b94c37ad6f44b1a88786bf/roles/ab816d03be024513884447335f2de35a => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:17:27.190582 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f2ce48-72d2-40e3-b4a7-1f958417a065 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:27.202447 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:27.366194 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f2ce48-72d2-40e3-b4a7-1f958417a065 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:27.420653 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4213245e-1a03-4988-abe4-5686ec980bb9 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:27.456883 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:27.471712 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d01065-95e7-4f9e-ad9a-5a6136049642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:27.472387 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d01065-95e7-4f9e-ad9a-5a6136049642 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:27.472670 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d01065-95e7-4f9e-ad9a-5a6136049642 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:27.473594 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d01065-95e7-4f9e-ad9a-5a6136049642 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:27.486120 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b47c50c-813a-4c9c-b9b1-22bed741265c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:27.486771 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b47c50c-813a-4c9c-b9b1-22bed741265c None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:27.487053 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b47c50c-813a-4c9c-b9b1-22bed741265c None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:27.487332 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b47c50c-813a-4c9c-b9b1-22bed741265c None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:27.488024 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b47c50c-813a-4c9c-b9b1-22bed741265c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:17:27.488333 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b47c50c-813a-4c9c-b9b1-22bed741265c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:27.490376 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b47c50c-813a-4c9c-b9b1-22bed741265c None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:27.494485 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d01065-95e7-4f9e-ad9a-5a6136049642 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:17:27.499371 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-73d01065-95e7-4f9e-ad9a-5a6136049642 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:17:27.499371 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:17:27.499371 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:17:27.499371 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:17:27.499371 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:17:27.499371 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:17:27.499371 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:17:27.499371 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:17:27.499371 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:17:27.499371 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:17:27.499371 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:17:27.499371 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:17:27.499371 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:17:27.499371 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:17:27.499371 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:17:27.499371 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:17:27.500191 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:17:27.500191 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:17:27.500191 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:17:27.500191 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:17:27.500191 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:17:27.500191 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:17:27.500191 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:17:27.500191 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:17:27.500191 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:17:27.500191 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:17:27.500191 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:17:27.500191 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:17:27.500191 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:17:27.500191 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:17:27.500191 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:17:27.500191 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:17:27.500995 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:17:27.500995 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:17:27.500995 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:17:27.500995 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:17:27.500995 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:17:27.500995 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:17:27.500995 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:17:27.500995 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:17:27.500995 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:17:27.500995 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:17:27.500995 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:17:27.500995 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:17:27.501585 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 83/159] 10.222.0.102 () {56 vars in 1116 bytes} [Thu Sep 1 17:17:27 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:27.501652 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 77/160] 10.222.0.102 () {56 vars in 1101 bytes} [Thu Sep 1 17:17:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 443 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:17:27.501712 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b47c50c-813a-4c9c-b9b1-22bed741265c None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:27.506228 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d01065-95e7-4f9e-ad9a-5a6136049642 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:27.620856 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:27.627361 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d01065-95e7-4f9e-ad9a-5a6136049642 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:27.664080 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b47c50c-813a-4c9c-b9b1-22bed741265c None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:27.703227 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fd4c3b-c97a-40e2-a584-17db66ab6940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:27.703847 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fd4c3b-c97a-40e2-a584-17db66ab6940 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:27.704136 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fd4c3b-c97a-40e2-a584-17db66ab6940 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:27.704413 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fd4c3b-c97a-40e2-a584-17db66ab6940 None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:27.705084 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fd4c3b-c97a-40e2-a584-17db66ab6940 None admin] RBAC: Adding query filter params (name=member) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:17:27.706112 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fd4c3b-c97a-40e2-a584-17db66ab6940 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:27.712108 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fd4c3b-c97a-40e2-a584-17db66ab6940 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:27.721074 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 84/161] 10.222.0.102 () {56 vars in 1106 bytes} [Thu Sep 1 17:17:27 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:27.721768 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fd4c3b-c97a-40e2-a584-17db66ab6940 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:27.722236 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dcc272-07f0-4cf9-9d36-51794aa43912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:27.723189 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dcc272-07f0-4cf9-9d36-51794aa43912 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:27.723673 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dcc272-07f0-4cf9-9d36-51794aa43912 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:27.724135 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dcc272-07f0-4cf9-9d36-51794aa43912 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:27.750663 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dcc272-07f0-4cf9-9d36-51794aa43912 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:17:27.756682 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-c0dcc272-07f0-4cf9-9d36-51794aa43912 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:17:27.756682 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:17:27.756682 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:17:27.756682 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:17:27.756682 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:17:27.756682 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:17:27.756682 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:17:27.756682 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:17:27.756682 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:17:27.756682 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:17:27.756682 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:17:27.756682 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:17:27.756682 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:17:27.756682 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:17:27.756682 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:17:27.756682 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:17:27.757560 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:17:27.757560 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:17:27.757560 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:17:27.757560 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:17:27.757560 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:17:27.757560 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:17:27.757560 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:17:27.757560 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:17:27.757560 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:17:27.757560 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:17:27.757560 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:17:27.757560 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:17:27.757560 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:17:27.757560 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:17:27.757560 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:17:27.757560 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:17:27.758677 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:17:27.758677 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:17:27.758677 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:17:27.758677 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:17:27.758677 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:17:27.758677 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:17:27.758677 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:17:27.758677 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:17:27.758677 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:17:27.758677 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:17:27.758677 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:17:27.758677 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:17:27.765367 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 78/162] 10.222.0.102 () {56 vars in 1104 bytes} [Thu Sep 1 17:17:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:17:27.766531 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dcc272-07f0-4cf9-9d36-51794aa43912 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:27.777689 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:27.873177 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fd4c3b-c97a-40e2-a584-17db66ab6940 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:27.935668 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79df06d-c7d5-4f12-a866-8d701008bc4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:27.936286 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79df06d-c7d5-4f12-a866-8d701008bc4e None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:27.936569 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79df06d-c7d5-4f12-a866-8d701008bc4e None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:27.936844 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79df06d-c7d5-4f12-a866-8d701008bc4e None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:27.937539 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79df06d-c7d5-4f12-a866-8d701008bc4e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:17:27.937848 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79df06d-c7d5-4f12-a866-8d701008bc4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:27.939303 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79df06d-c7d5-4f12-a866-8d701008bc4e None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:27.952976 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 85/163] 10.222.0.102 () {56 vars in 1104 bytes} [Thu Sep 1 17:17:27 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:27.953864 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79df06d-c7d5-4f12-a866-8d701008bc4e None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:27.961509 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dcc272-07f0-4cf9-9d36-51794aa43912 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:28.022527 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287ef33e-f2fc-49f4-9a75-9db6051fb174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:28.023172 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287ef33e-f2fc-49f4-9a75-9db6051fb174 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:28.023451 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287ef33e-f2fc-49f4-9a75-9db6051fb174 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:28.023724 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287ef33e-f2fc-49f4-9a75-9db6051fb174 None admin] PATH_INFO: `/v3/users/27f309e66b2c4003a820573ada1b0958` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:28.029102 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287ef33e-f2fc-49f4-9a75-9db6051fb174 None admin] RBAC: Authorizing `identity:get_user(user_id=27f309e66b2c4003a820573ada1b0958)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:28.030712 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287ef33e-f2fc-49f4-9a75-9db6051fb174 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:28.032351 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 79/164] 10.222.0.102 () {56 vars in 1182 bytes} [Thu Sep 1 17:17:27 2022] GET /identity/v3/users/27f309e66b2c4003a820573ada1b0958 => generated 276 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:28.033128 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287ef33e-f2fc-49f4-9a75-9db6051fb174 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:28.074984 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:28.140926 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79df06d-c7d5-4f12-a866-8d701008bc4e None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:28.159992 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:28.178302 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287ef33e-f2fc-49f4-9a75-9db6051fb174 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:28.202953 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519e0bab-7e19-4b74-9a88-4adbf74fe1f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:28.203601 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519e0bab-7e19-4b74-9a88-4adbf74fe1f9 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:28.204079 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519e0bab-7e19-4b74-9a88-4adbf74fe1f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:28.204344 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519e0bab-7e19-4b74-9a88-4adbf74fe1f9 None admin] PATH_INFO: `/v3/users/27f309e66b2c4003a820573ada1b0958` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:28.206384 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519e0bab-7e19-4b74-9a88-4adbf74fe1f9 None admin] RBAC: Authorizing `identity:get_user(user_id=27f309e66b2c4003a820573ada1b0958)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:28.207893 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519e0bab-7e19-4b74-9a88-4adbf74fe1f9 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:28.209523 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 86/165] 10.222.0.102 () {56 vars in 1182 bytes} [Thu Sep 1 17:17:27 2022] GET /identity/v3/users/27f309e66b2c4003a820573ada1b0958 => generated 276 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:28.210290 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519e0bab-7e19-4b74-9a88-4adbf74fe1f9 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:28.244338 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b318d22-7296-40f3-8942-6a3b552b37ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:28.244954 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b318d22-7296-40f3-8942-6a3b552b37ce None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:28.245233 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b318d22-7296-40f3-8942-6a3b552b37ce None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:28.245525 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b318d22-7296-40f3-8942-6a3b552b37ce None admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:28.246193 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b318d22-7296-40f3-8942-6a3b552b37ce None admin] RBAC: Adding query filter params (name=member) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:17:28.246494 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b318d22-7296-40f3-8942-6a3b552b37ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:28.247868 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b318d22-7296-40f3-8942-6a3b552b37ce None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:28.258706 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 80/166] 10.222.0.102 () {56 vars in 1106 bytes} [Thu Sep 1 17:17:28 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:28.265609 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b318d22-7296-40f3-8942-6a3b552b37ce None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:28.339416 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519e0bab-7e19-4b74-9a88-4adbf74fe1f9 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:28.376071 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe9de8e-dfe1-4ad5-9669-a60ec7b856a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:28.376859 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe9de8e-dfe1-4ad5-9669-a60ec7b856a2 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:28.377118 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe9de8e-dfe1-4ad5-9669-a60ec7b856a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:28.377470 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe9de8e-dfe1-4ad5-9669-a60ec7b856a2 None admin] PATH_INFO: `/v3/users/e6c9197e0f844432a03b9e75fc87712b` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:28.378613 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe9de8e-dfe1-4ad5-9669-a60ec7b856a2 None admin] RBAC: Authorizing `identity:get_user(user_id=e6c9197e0f844432a03b9e75fc87712b)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:28.380608 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe9de8e-dfe1-4ad5-9669-a60ec7b856a2 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:28.382560 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 87/167] 10.222.0.102 () {56 vars in 1182 bytes} [Thu Sep 1 17:17:28 2022] GET /identity/v3/users/e6c9197e0f844432a03b9e75fc87712b => generated 248 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:28.383724 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c8e194-7463-44a1-9513-bbc71c6dbb37 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:28.388908 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c8e194-7463-44a1-9513-bbc71c6dbb37 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:28.389249 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c8e194-7463-44a1-9513-bbc71c6dbb37 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:28.391183 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 88/168] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:17:28 2022] GET /identity => generated 271 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:17:28.391924 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c8e194-7463-44a1-9513-bbc71c6dbb37 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:28.475058 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b318d22-7296-40f3-8942-6a3b552b37ce None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:28.477688 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:28.505666 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:28.522320 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20adab9-44f7-4c82-8846-0cbd1173f2f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:28.523337 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20adab9-44f7-4c82-8846-0cbd1173f2f0 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:28.523816 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20adab9-44f7-4c82-8846-0cbd1173f2f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:28.524287 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20adab9-44f7-4c82-8846-0cbd1173f2f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:28.591644 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c8e194-7463-44a1-9513-bbc71c6dbb37 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:28.631855 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b96798-d6ca-40b2-8c38-6474b59aa855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:28.632655 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b96798-d6ca-40b2-8c38-6474b59aa855 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:28.632932 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b96798-d6ca-40b2-8c38-6474b59aa855 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:28.633202 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b96798-d6ca-40b2-8c38-6474b59aa855 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:28.665121 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20adab9-44f7-4c82-8846-0cbd1173f2f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:28.667225 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20adab9-44f7-4c82-8846-0cbd1173f2f0 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:28.766460 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b96798-d6ca-40b2-8c38-6474b59aa855 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:28.767958 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b96798-d6ca-40b2-8c38-6474b59aa855 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:28.777693 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:28.787041 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 81/169] 10.222.0.102 () {58 vars in 1287 bytes} [Thu Sep 1 17:17:28 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 528 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 17:17:28.788230 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20adab9-44f7-4c82-8846-0cbd1173f2f0 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:28.821440 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:28.884246 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 89/170] 10.222.0.102 () {58 vars in 1287 bytes} [Thu Sep 1 17:17:28 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 17:17:28.884929 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b96798-d6ca-40b2-8c38-6474b59aa855 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:28.904945 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20adab9-44f7-4c82-8846-0cbd1173f2f0 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:28.943764 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba66130-3ed0-4432-803a-fbc5f9e93e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:28.944756 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba66130-3ed0-4432-803a-fbc5f9e93e71 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:28.945267 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba66130-3ed0-4432-803a-fbc5f9e93e71 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:28.945903 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba66130-3ed0-4432-803a-fbc5f9e93e71 None admin] PATH_INFO: `/v3/users/27f309e66b2c4003a820573ada1b0958` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:28.947616 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba66130-3ed0-4432-803a-fbc5f9e93e71 None admin] RBAC: Authorizing `identity:get_user(user_id=27f309e66b2c4003a820573ada1b0958)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:28.949064 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba66130-3ed0-4432-803a-fbc5f9e93e71 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:28.950990 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 82/171] 10.222.0.102 () {56 vars in 1182 bytes} [Thu Sep 1 17:17:28 2022] GET /identity/v3/users/27f309e66b2c4003a820573ada1b0958 => generated 276 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:28.951970 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dabcfa-62e1-428a-9fc1-278bcad4374c None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:28.952253 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dabcfa-62e1-428a-9fc1-278bcad4374c None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:28.952634 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dabcfa-62e1-428a-9fc1-278bcad4374c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:28.970873 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-86dabcfa-62e1-428a-9fc1-278bcad4374c None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:17:28.972078 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-86dabcfa-62e1-428a-9fc1-278bcad4374c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:17:28.995792 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b96798-d6ca-40b2-8c38-6474b59aa855 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:29.024293 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:29.035023 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fabfb18-ad4a-4229-8c35-d506d72d038a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:29.036434 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fabfb18-ad4a-4229-8c35-d506d72d038a None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:29.036945 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fabfb18-ad4a-4229-8c35-d506d72d038a None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:29.037588 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fabfb18-ad4a-4229-8c35-d506d72d038a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:29.042265 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 83/172] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:17:28 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:17:29.043562 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf040373-91ab-4107-be2c-1fde2c77d3ed None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:29.043856 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf040373-91ab-4107-be2c-1fde2c77d3ed None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:29.044129 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf040373-91ab-4107-be2c-1fde2c77d3ed None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:29.045548 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 84/173] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:17:29 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:17:29.046151 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf040373-91ab-4107-be2c-1fde2c77d3ed None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:29.065785 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:29.114658 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fabfb18-ad4a-4229-8c35-d506d72d038a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:29.116557 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fabfb18-ad4a-4229-8c35-d506d72d038a None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:29.172691 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf040373-91ab-4107-be2c-1fde2c77d3ed None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:29.209469 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22820e8f-95c3-4a5a-88e7-4c6960b643f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:29.210326 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22820e8f-95c3-4a5a-88e7-4c6960b643f4 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:29.210812 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22820e8f-95c3-4a5a-88e7-4c6960b643f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:29.211226 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22820e8f-95c3-4a5a-88e7-4c6960b643f4 None admin] PATH_INFO: `/v3/projects/8e88c67b942f45108c95a833547f9bc8` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:29.212555 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22820e8f-95c3-4a5a-88e7-4c6960b643f4 None admin] RBAC: Authorizing `identity:get_project(project_id=8e88c67b942f45108c95a833547f9bc8)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:29.214168 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22820e8f-95c3-4a5a-88e7-4c6960b643f4 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:29.216322 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 85/174] 10.222.0.102 () {56 vars in 1191 bytes} [Thu Sep 1 17:17:29 2022] GET /identity/v3/projects/8e88c67b942f45108c95a833547f9bc8 => generated 299 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:29.217404 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 90/175] 10.222.0.102 () {58 vars in 1287 bytes} [Thu Sep 1 17:17:28 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 17:17:29.217775 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22820e8f-95c3-4a5a-88e7-4c6960b643f4 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:29.218261 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fabfb18-ad4a-4229-8c35-d506d72d038a None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:29.265688 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:29.274249 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:29.322160 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fabfb18-ad4a-4229-8c35-d506d72d038a None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:29.330899 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22820e8f-95c3-4a5a-88e7-4c6960b643f4 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:29.360463 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bae6a9f-7504-488d-b753-d7ae946a215b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:29.361184 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bae6a9f-7504-488d-b753-d7ae946a215b None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:29.361523 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bae6a9f-7504-488d-b753-d7ae946a215b None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:29.361962 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bae6a9f-7504-488d-b753-d7ae946a215b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:29.366419 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef7713a-d612-4cfb-9717-0ad7565e3e00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:29.367049 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef7713a-d612-4cfb-9717-0ad7565e3e00 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:29.367325 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef7713a-d612-4cfb-9717-0ad7565e3e00 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:29.367600 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef7713a-d612-4cfb-9717-0ad7565e3e00 None admin] PATH_INFO: `/v3/projects/dc1ae957803645b4abe7503e4399bf10` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:29.368764 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef7713a-d612-4cfb-9717-0ad7565e3e00 None admin] RBAC: Authorizing `identity:get_project(project_id=dc1ae957803645b4abe7503e4399bf10)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:29.370253 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef7713a-d612-4cfb-9717-0ad7565e3e00 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:29.373361 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 86/176] 10.222.0.102 () {56 vars in 1191 bytes} [Thu Sep 1 17:17:29 2022] GET /identity/v3/projects/dc1ae957803645b4abe7503e4399bf10 => generated 313 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:29.373361 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f283ee8d-3467-4f4f-8705-40abe37112be None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:29.373903 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f283ee8d-3467-4f4f-8705-40abe37112be None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:29.374320 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f283ee8d-3467-4f4f-8705-40abe37112be None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:29.379856 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 87/177] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:17:29 2022] GET /identity => generated 271 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:17:29.381217 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3cf3bf-7959-4a99-b860-f075e0943fad None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:29.381531 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3cf3bf-7959-4a99-b860-f075e0943fad None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:29.381810 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3cf3bf-7959-4a99-b860-f075e0943fad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:29.401058 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-1b3cf3bf-7959-4a99-b860-f075e0943fad None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:17:29.402291 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-1b3cf3bf-7959-4a99-b860-f075e0943fad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:17:29.445068 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bae6a9f-7504-488d-b753-d7ae946a215b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:29.448143 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bae6a9f-7504-488d-b753-d7ae946a215b None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:29.485365 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 88/178] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:17:29 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:17:29.485365 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3cf3bf-7959-4a99-b860-f075e0943fad None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:29.512821 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:29.529103 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:29.587295 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3cf3bf-7959-4a99-b860-f075e0943fad None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:29.593268 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 91/179] 10.222.0.102 () {58 vars in 1287 bytes} [Thu Sep 1 17:17:29 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 17:17:29.594158 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bae6a9f-7504-488d-b753-d7ae946a215b None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:29.632121 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4244dac7-6ecd-4782-b855-44a917ceb145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:29.632855 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4244dac7-6ecd-4782-b855-44a917ceb145 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:29.633136 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4244dac7-6ecd-4782-b855-44a917ceb145 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:29.633539 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4244dac7-6ecd-4782-b855-44a917ceb145 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:29.634273 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4244dac7-6ecd-4782-b855-44a917ceb145 None admin] RBAC: Adding query filter params (role.id=abef78c185c74c16a2c7e8a170583f9f, scope.project.id=8e88c67b942f45108c95a833547f9bc8, user.id=27f309e66b2c4003a820573ada1b0958) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:17:29.634580 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4244dac7-6ecd-4782-b855-44a917ceb145 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:29.636148 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4244dac7-6ecd-4782-b855-44a917ceb145 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:29.642570 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 89/180] 10.222.0.102 () {56 vars in 1379 bytes} [Thu Sep 1 17:17:29 2022] GET /identity/v3/role_assignments?user.id=27f309e66b2c4003a820573ada1b0958&scope.project.id=8e88c67b942f45108c95a833547f9bc8&role.id=abef78c185c74c16a2c7e8a170583f9f => generated 610 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:29.650663 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4244dac7-6ecd-4782-b855-44a917ceb145 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:29.721602 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bae6a9f-7504-488d-b753-d7ae946a215b None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:29.764501 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6fd023-35c2-41a7-883d-0fe8acd31d50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:29.765071 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6fd023-35c2-41a7-883d-0fe8acd31d50 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:29.765402 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6fd023-35c2-41a7-883d-0fe8acd31d50 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:29.765687 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6fd023-35c2-41a7-883d-0fe8acd31d50 None admin] PATH_INFO: `/v3/projects/8e88c67b942f45108c95a833547f9bc8` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:29.766850 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6fd023-35c2-41a7-883d-0fe8acd31d50 None admin] RBAC: Authorizing `identity:get_project(project_id=8e88c67b942f45108c95a833547f9bc8)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:29.768614 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6fd023-35c2-41a7-883d-0fe8acd31d50 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:29.770558 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 92/181] 10.222.0.102 () {56 vars in 1191 bytes} [Thu Sep 1 17:17:29 2022] GET /identity/v3/projects/8e88c67b942f45108c95a833547f9bc8 => generated 299 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:29.771346 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6fd023-35c2-41a7-883d-0fe8acd31d50 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:29.789999 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:29.866052 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4244dac7-6ecd-4782-b855-44a917ceb145 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:29.888548 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:29.899523 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6fd023-35c2-41a7-883d-0fe8acd31d50 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:29.919961 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8626e314-c744-4e75-b927-06cfc0c01bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:29.920706 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8626e314-c744-4e75-b927-06cfc0c01bc5 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:29.920952 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8626e314-c744-4e75-b927-06cfc0c01bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:29.921228 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8626e314-c744-4e75-b927-06cfc0c01bc5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:29.936111 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8626e314-c744-4e75-b927-06cfc0c01bc5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:17:29.942291 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-8626e314-c744-4e75-b927-06cfc0c01bc5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:17:29.942291 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:17:29.942291 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:17:29.942291 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:17:29.942291 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:17:29.942291 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:17:29.942291 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:17:29.942291 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:17:29.942291 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:17:29.942291 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:17:29.942291 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:17:29.942291 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:17:29.942291 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:17:29.942291 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:17:29.942291 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:17:29.942291 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:17:29.943239 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:17:29.943239 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:17:29.943239 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:17:29.943239 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:17:29.943239 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:17:29.943239 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:17:29.943239 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:17:29.943239 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:17:29.943239 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:17:29.943239 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:17:29.943239 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:17:29.943239 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:17:29.943239 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:17:29.943239 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:17:29.943239 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:17:29.943239 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:17:29.944045 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:17:29.944045 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:17:29.944045 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:17:29.944045 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:17:29.944045 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:17:29.944045 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:17:29.944045 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:17:29.944045 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:17:29.944045 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:17:29.944045 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:17:29.944045 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:17:29.944045 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:17:29.944654 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 90/182] 10.222.0.102 () {56 vars in 1101 bytes} [Thu Sep 1 17:17:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:17:29.945712 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00f199f-6c22-4a4d-a0c3-de3530a2eb2c None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:29.945996 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00f199f-6c22-4a4d-a0c3-de3530a2eb2c None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:29.946272 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00f199f-6c22-4a4d-a0c3-de3530a2eb2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:29.954029 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a99274-5dff-4650-9360-04aea1436110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:29.954664 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a99274-5dff-4650-9360-04aea1436110 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:29.954945 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a99274-5dff-4650-9360-04aea1436110 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:29.955218 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a99274-5dff-4650-9360-04aea1436110 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:29.955966 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a99274-5dff-4650-9360-04aea1436110 None admin] RBAC: Adding query filter params (role.id=b75cb360925e4ba5a65f14965c103ffd, scope.project.id=dc1ae957803645b4abe7503e4399bf10, user.id=27f309e66b2c4003a820573ada1b0958) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:17:29.956280 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a99274-5dff-4650-9360-04aea1436110 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:29.958333 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a99274-5dff-4650-9360-04aea1436110 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:29.965879 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 93/183] 10.222.0.102 () {56 vars in 1379 bytes} [Thu Sep 1 17:17:29 2022] GET /identity/v3/role_assignments?user.id=27f309e66b2c4003a820573ada1b0958&scope.project.id=dc1ae957803645b4abe7503e4399bf10&role.id=b75cb360925e4ba5a65f14965c103ffd => generated 610 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:29.966718 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a99274-5dff-4650-9360-04aea1436110 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:29.968612 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-b00f199f-6c22-4a4d-a0c3-de3530a2eb2c None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:17:29.969857 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-b00f199f-6c22-4a4d-a0c3-de3530a2eb2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:17:30.054050 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 91/184] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:17:29 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:17:30.055144 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00f199f-6c22-4a4d-a0c3-de3530a2eb2c None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:30.072263 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:30.113008 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a99274-5dff-4650-9360-04aea1436110 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:30.161389 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bff623f-a0b8-4468-8285-1cb8b69c178e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:30.162342 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bff623f-a0b8-4468-8285-1cb8b69c178e None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:30.162774 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bff623f-a0b8-4468-8285-1cb8b69c178e None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:30.163242 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bff623f-a0b8-4468-8285-1cb8b69c178e None admin] PATH_INFO: `/v3/projects/8e88c67b942f45108c95a833547f9bc8` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:30.164942 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bff623f-a0b8-4468-8285-1cb8b69c178e None admin] RBAC: Authorizing `identity:get_project(project_id=8e88c67b942f45108c95a833547f9bc8)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:30.166666 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bff623f-a0b8-4468-8285-1cb8b69c178e None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:30.168549 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 94/185] 10.222.0.102 () {56 vars in 1191 bytes} [Thu Sep 1 17:17:29 2022] GET /identity/v3/projects/8e88c67b942f45108c95a833547f9bc8 => generated 299 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:30.169484 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bff623f-a0b8-4468-8285-1cb8b69c178e None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:30.190970 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00f199f-6c22-4a4d-a0c3-de3530a2eb2c None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:30.196064 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:30.244567 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b47426c-bc17-4ea8-b13b-a185b8ed7be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:30.245182 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b47426c-bc17-4ea8-b13b-a185b8ed7be3 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:30.245538 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b47426c-bc17-4ea8-b13b-a185b8ed7be3 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:30.245819 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b47426c-bc17-4ea8-b13b-a185b8ed7be3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:30.246527 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b47426c-bc17-4ea8-b13b-a185b8ed7be3 None admin] RBAC: Adding query filter params (role.id=36577a6f46d642d4b81805ede9872490, scope.project.id=8e88c67b942f45108c95a833547f9bc8, user.id=e6c9197e0f844432a03b9e75fc87712b) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:17:30.246832 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b47426c-bc17-4ea8-b13b-a185b8ed7be3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:30.248248 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b47426c-bc17-4ea8-b13b-a185b8ed7be3 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:30.261339 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 92/186] 10.222.0.102 () {56 vars in 1379 bytes} [Thu Sep 1 17:17:30 2022] GET /identity/v3/role_assignments?user.id=e6c9197e0f844432a03b9e75fc87712b&scope.project.id=8e88c67b942f45108c95a833547f9bc8&role.id=36577a6f46d642d4b81805ede9872490 => generated 610 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:30.261339 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fdcf02-7903-4383-b599-c430d4d73bef None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:30.261339 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fdcf02-7903-4383-b599-c430d4d73bef None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:30.261339 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fdcf02-7903-4383-b599-c430d4d73bef None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:30.261339 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 93/187] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:17:30 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:17:30.261919 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fdcf02-7903-4383-b599-c430d4d73bef None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:30.313223 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bff623f-a0b8-4468-8285-1cb8b69c178e None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:30.356412 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffdea7e-7a77-41cb-9a0f-761f80f82821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:30.357062 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffdea7e-7a77-41cb-9a0f-761f80f82821 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:30.357374 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffdea7e-7a77-41cb-9a0f-761f80f82821 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:30.357648 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffdea7e-7a77-41cb-9a0f-761f80f82821 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:30.370446 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffdea7e-7a77-41cb-9a0f-761f80f82821 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:17:30.375905 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-4ffdea7e-7a77-41cb-9a0f-761f80f82821 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 17:17:30.375905 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:17:30.375905 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:17:30.375905 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:17:30.375905 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:17:30.375905 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:17:30.375905 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:17:30.375905 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:17:30.375905 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:17:30.375905 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:17:30.375905 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:17:30.375905 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:17:30.375905 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:17:30.375905 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:17:30.375905 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:17:30.375905 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:17:30.376857 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:17:30.376857 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:17:30.376857 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:17:30.376857 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:17:30.376857 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:17:30.376857 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:17:30.376857 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:17:30.376857 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:17:30.376857 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:17:30.376857 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:17:30.376857 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:17:30.376857 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:17:30.376857 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:17:30.376857 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:17:30.376857 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:17:30.376857 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:17:30.377713 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:17:30.377713 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:17:30.377713 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:17:30.377713 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:17:30.377713 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:17:30.377713 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:17:30.377713 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:17:30.377713 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:17:30.377713 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:17:30.377713 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:17:30.377713 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 17:17:30.377713 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:17:30.377713 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 95/188] 10.222.0.102 () {56 vars in 1119 bytes} [Thu Sep 1 17:17:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:17:30.378488 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffdea7e-7a77-41cb-9a0f-761f80f82821 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:30.384690 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:30.410057 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fdcf02-7903-4383-b599-c430d4d73bef None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:30.455362 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4421fde1-a58b-478e-8d96-ca111f28765c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:30.456431 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4421fde1-a58b-478e-8d96-ca111f28765c None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:30.456906 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4421fde1-a58b-478e-8d96-ca111f28765c None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:30.458232 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4421fde1-a58b-478e-8d96-ca111f28765c None admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:30.458703 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4421fde1-a58b-478e-8d96-ca111f28765c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:17:30.461191 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4421fde1-a58b-478e-8d96-ca111f28765c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:30.461191 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4421fde1-a58b-478e-8d96-ca111f28765c None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:30.475008 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 94/189] 10.222.0.102 () {56 vars in 1104 bytes} [Thu Sep 1 17:17:30 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:30.476336 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e7ed8a-051d-4049-8359-f2a662e1fb98 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:30.476543 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e7ed8a-051d-4049-8359-f2a662e1fb98 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:30.477340 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e7ed8a-051d-4049-8359-f2a662e1fb98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:30.506098 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-c7e7ed8a-051d-4049-8359-f2a662e1fb98 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:17:30.507314 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-c7e7ed8a-051d-4049-8359-f2a662e1fb98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:17:30.507419 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffdea7e-7a77-41cb-9a0f-761f80f82821 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:30.539188 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:30.548558 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a5c98f-b74f-4cd9-a288-42f48616ba78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:30.549166 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a5c98f-b74f-4cd9-a288-42f48616ba78 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:30.549477 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a5c98f-b74f-4cd9-a288-42f48616ba78 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:30.549749 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a5c98f-b74f-4cd9-a288-42f48616ba78 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:30.550478 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a5c98f-b74f-4cd9-a288-42f48616ba78 None admin] RBAC: Adding query filter params (role.id=b75cb360925e4ba5a65f14965c103ffd, scope.project.id=8e88c67b942f45108c95a833547f9bc8, user.id=27f309e66b2c4003a820573ada1b0958) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:17:30.550784 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a5c98f-b74f-4cd9-a288-42f48616ba78 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:30.552750 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a5c98f-b74f-4cd9-a288-42f48616ba78 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:30.558398 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 96/190] 10.222.0.102 () {56 vars in 1379 bytes} [Thu Sep 1 17:17:30 2022] GET /identity/v3/role_assignments?user.id=27f309e66b2c4003a820573ada1b0958&scope.project.id=8e88c67b942f45108c95a833547f9bc8&role.id=b75cb360925e4ba5a65f14965c103ffd => generated 610 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:30.559193 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a5c98f-b74f-4cd9-a288-42f48616ba78 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:30.592723 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 95/191] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:17:30 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:17:30.599403 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e7ed8a-051d-4049-8359-f2a662e1fb98 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:30.662633 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:30.721604 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a5c98f-b74f-4cd9-a288-42f48616ba78 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:30.746629 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e7ed8a-051d-4049-8359-f2a662e1fb98 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:30.776041 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9f7763-26ba-436b-8c34-72ce72f046a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:30.777013 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9f7763-26ba-436b-8c34-72ce72f046a6 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:30.777512 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9f7763-26ba-436b-8c34-72ce72f046a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:30.777967 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9f7763-26ba-436b-8c34-72ce72f046a6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:30.779276 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9f7763-26ba-436b-8c34-72ce72f046a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:30.780941 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9f7763-26ba-436b-8c34-72ce72f046a6 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:30.782751 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 97/192] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:17:30 2022] GET /identity/v3/domains/default => generated 200 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:30.783698 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9f7763-26ba-436b-8c34-72ce72f046a6 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:30.789208 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e297618c-ed04-4267-8e4a-f54610ffd229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:30.790151 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e297618c-ed04-4267-8e4a-f54610ffd229 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:30.790621 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e297618c-ed04-4267-8e4a-f54610ffd229 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:30.790886 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e297618c-ed04-4267-8e4a-f54610ffd229 None admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:30.791688 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e297618c-ed04-4267-8e4a-f54610ffd229 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:17:30.791987 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e297618c-ed04-4267-8e4a-f54610ffd229 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:30.793438 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e297618c-ed04-4267-8e4a-f54610ffd229 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:30.803206 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 96/193] 10.222.0.102 () {56 vars in 1116 bytes} [Thu Sep 1 17:17:30 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:30.804033 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e297618c-ed04-4267-8e4a-f54610ffd229 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:30.839115 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:30.906391 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9f7763-26ba-436b-8c34-72ce72f046a6 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:30.917403 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e297618c-ed04-4267-8e4a-f54610ffd229 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:30.941263 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a263aa45-1cbd-4d4b-905f-ba1dd8586a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:30.941943 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a263aa45-1cbd-4d4b-905f-ba1dd8586a62 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:30.942230 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a263aa45-1cbd-4d4b-905f-ba1dd8586a62 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:30.942502 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a263aa45-1cbd-4d4b-905f-ba1dd8586a62 None admin] PATH_INFO: `/v3/users/7cf4904211ee414793d522ae570a016e` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:30.944358 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:30.950658 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b4279c-4d8e-466c-afe9-2b42cdb290cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:30.951263 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b4279c-4d8e-466c-afe9-2b42cdb290cb None admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:30.951536 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b4279c-4d8e-466c-afe9-2b42cdb290cb None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:30.951806 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b4279c-4d8e-466c-afe9-2b42cdb290cb None admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:30.952658 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b4279c-4d8e-466c-afe9-2b42cdb290cb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:30.954200 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b4279c-4d8e-466c-afe9-2b42cdb290cb None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:30.956580 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a263aa45-1cbd-4d4b-905f-ba1dd8586a62 None admin] RBAC: Authorizing `identity:get_user(user_id=7cf4904211ee414793d522ae570a016e)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:30.958548 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a263aa45-1cbd-4d4b-905f-ba1dd8586a62 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:30.960211 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 98/194] 10.222.0.102 () {56 vars in 1182 bytes} [Thu Sep 1 17:17:30 2022] GET /identity/v3/users/7cf4904211ee414793d522ae570a016e => generated 284 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:30.965210 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a263aa45-1cbd-4d4b-905f-ba1dd8586a62 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:30.975548 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 97/195] 10.222.0.102 () {60 vars in 1137 bytes} [Thu Sep 1 17:17:30 2022] POST /identity/v3/users => generated 298 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:17:30.976361 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b4279c-4d8e-466c-afe9-2b42cdb290cb None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:31.075022 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a263aa45-1cbd-4d4b-905f-ba1dd8586a62 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:31.084007 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:31.113350 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b4279c-4d8e-466c-afe9-2b42cdb290cb None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:31.115223 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8485476-b01a-4aa3-9446-949e2e68aad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:31.115865 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8485476-b01a-4aa3-9446-949e2e68aad0 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:31.116140 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8485476-b01a-4aa3-9446-949e2e68aad0 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:31.117097 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8485476-b01a-4aa3-9446-949e2e68aad0 None admin] PATH_INFO: `/v3/users/7cf4904211ee414793d522ae570a016e` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:31.117657 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8485476-b01a-4aa3-9446-949e2e68aad0 None admin] RBAC: Authorizing `identity:get_user(user_id=7cf4904211ee414793d522ae570a016e)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:31.119110 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8485476-b01a-4aa3-9446-949e2e68aad0 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:31.126111 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 99/196] 10.222.0.102 () {56 vars in 1182 bytes} [Thu Sep 1 17:17:30 2022] GET /identity/v3/users/7cf4904211ee414793d522ae570a016e => generated 284 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:31.126111 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8485476-b01a-4aa3-9446-949e2e68aad0 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:31.154913 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68e0ea3-3445-46fa-8cd3-fddaa754176e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:31.155549 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68e0ea3-3445-46fa-8cd3-fddaa754176e None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:31.155829 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68e0ea3-3445-46fa-8cd3-fddaa754176e None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:31.156112 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68e0ea3-3445-46fa-8cd3-fddaa754176e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:31.169950 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68e0ea3-3445-46fa-8cd3-fddaa754176e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:17:31.176107 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-d68e0ea3-3445-46fa-8cd3-fddaa754176e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 17:17:31.176107 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:17:31.176107 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:17:31.176107 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:17:31.176107 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:17:31.176107 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:17:31.176107 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:17:31.176107 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:17:31.176107 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:17:31.176107 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:17:31.176107 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:17:31.176107 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:17:31.176107 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:17:31.176107 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:17:31.176107 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:17:31.176107 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:17:31.177172 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:17:31.177172 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:17:31.177172 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:17:31.177172 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:17:31.177172 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:17:31.177172 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:17:31.177172 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:17:31.177172 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:17:31.177172 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:17:31.177172 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:17:31.177172 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:17:31.177172 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:17:31.177172 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:17:31.177172 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:17:31.177172 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:17:31.177172 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:17:31.182005 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:17:31.182005 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:17:31.182005 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:17:31.182005 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:17:31.182005 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:17:31.182005 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:17:31.182005 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:17:31.182005 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:17:31.182005 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:17:31.182005 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:17:31.182005 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 17:17:31.182005 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:17:31.182005 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 98/197] 10.222.0.102 () {56 vars in 1104 bytes} [Thu Sep 1 17:17:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:17:31.182005 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68e0ea3-3445-46fa-8cd3-fddaa754176e None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:31.202508 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:31.264698 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8485476-b01a-4aa3-9446-949e2e68aad0 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:31.305384 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68e0ea3-3445-46fa-8cd3-fddaa754176e None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:31.310052 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e5255b-7dee-4ee9-81c8-e6a447abb145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:31.310733 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e5255b-7dee-4ee9-81c8-e6a447abb145 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:31.311070 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e5255b-7dee-4ee9-81c8-e6a447abb145 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:31.311364 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e5255b-7dee-4ee9-81c8-e6a447abb145 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:31.355648 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa959ec7-4797-4354-bdd9-56a4894f96b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:31.356277 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa959ec7-4797-4354-bdd9-56a4894f96b2 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:31.356558 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa959ec7-4797-4354-bdd9-56a4894f96b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:31.356829 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa959ec7-4797-4354-bdd9-56a4894f96b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:31.372234 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:31.396336 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e5255b-7dee-4ee9-81c8-e6a447abb145 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:31.397907 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e5255b-7dee-4ee9-81c8-e6a447abb145 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:31.454772 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa959ec7-4797-4354-bdd9-56a4894f96b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:31.456727 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa959ec7-4797-4354-bdd9-56a4894f96b2 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:31.462179 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:31.525282 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 100/198] 10.222.0.102 () {58 vars in 1287 bytes} [Thu Sep 1 17:17:31 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 17:17:31.526692 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e5255b-7dee-4ee9-81c8-e6a447abb145 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:31.565834 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 99/199] 10.222.0.102 () {58 vars in 1287 bytes} [Thu Sep 1 17:17:31 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 17:17:31.570291 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa959ec7-4797-4354-bdd9-56a4894f96b2 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:31.615577 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:31.673630 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e5255b-7dee-4ee9-81c8-e6a447abb145 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:31.681610 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa959ec7-4797-4354-bdd9-56a4894f96b2 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:31.718855 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0c0672-443e-4e7d-8bda-da3ce1a5351c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:31.719061 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909443f0-5dc4-4c43-abd9-3d9aa7c4749e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:31.719656 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0c0672-443e-4e7d-8bda-da3ce1a5351c None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:31.719768 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909443f0-5dc4-4c43-abd9-3d9aa7c4749e None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:31.719944 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909443f0-5dc4-4c43-abd9-3d9aa7c4749e None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:31.720184 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0c0672-443e-4e7d-8bda-da3ce1a5351c None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:31.720302 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909443f0-5dc4-4c43-abd9-3d9aa7c4749e None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:31.720672 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0c0672-443e-4e7d-8bda-da3ce1a5351c None admin] PATH_INFO: `/v3/projects/3325fd0bf7e2439aa626201e2ae25c6b` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:31.720871 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909443f0-5dc4-4c43-abd9-3d9aa7c4749e None admin] RBAC: Adding query filter params (name=reader) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:17:31.721178 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909443f0-5dc4-4c43-abd9-3d9aa7c4749e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:31.722082 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0c0672-443e-4e7d-8bda-da3ce1a5351c None admin] RBAC: Authorizing `identity:get_project(project_id=3325fd0bf7e2439aa626201e2ae25c6b)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:31.722610 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909443f0-5dc4-4c43-abd9-3d9aa7c4749e None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:31.724377 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0c0672-443e-4e7d-8bda-da3ce1a5351c None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:31.726155 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 100/200] 10.222.0.102 () {56 vars in 1191 bytes} [Thu Sep 1 17:17:31 2022] GET /identity/v3/projects/3325fd0bf7e2439aa626201e2ae25c6b => generated 303 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:31.727403 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0c0672-443e-4e7d-8bda-da3ce1a5351c None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:31.733775 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 101/201] 10.222.0.102 () {56 vars in 1106 bytes} [Thu Sep 1 17:17:31 2022] GET /identity/v3/roles?name=reader => generated 343 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:31.734685 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909443f0-5dc4-4c43-abd9-3d9aa7c4749e None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:31.761494 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:31.835669 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0c0672-443e-4e7d-8bda-da3ce1a5351c None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:31.843378 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:31.858071 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909443f0-5dc4-4c43-abd9-3d9aa7c4749e None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:31.875934 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6150a7-3220-44ae-94a6-366cedc42b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:31.876744 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6150a7-3220-44ae-94a6-366cedc42b53 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:31.877344 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6150a7-3220-44ae-94a6-366cedc42b53 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:31.877634 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6150a7-3220-44ae-94a6-366cedc42b53 None admin] PATH_INFO: `/v3/projects/3325fd0bf7e2439aa626201e2ae25c6b` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:31.878836 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6150a7-3220-44ae-94a6-366cedc42b53 None admin] RBAC: Authorizing `identity:get_project(project_id=3325fd0bf7e2439aa626201e2ae25c6b)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:31.880874 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6150a7-3220-44ae-94a6-366cedc42b53 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:31.887770 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 101/202] 10.222.0.102 () {56 vars in 1191 bytes} [Thu Sep 1 17:17:31 2022] GET /identity/v3/projects/3325fd0bf7e2439aa626201e2ae25c6b => generated 303 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:31.887770 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6150a7-3220-44ae-94a6-366cedc42b53 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:31.902725 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4919016-476f-4167-8ee2-85a34e845ff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:31.903633 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4919016-476f-4167-8ee2-85a34e845ff7 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:31.904085 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4919016-476f-4167-8ee2-85a34e845ff7 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:31.904607 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4919016-476f-4167-8ee2-85a34e845ff7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:31.905560 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4919016-476f-4167-8ee2-85a34e845ff7 None admin] RBAC: Adding query filter params (role.id=36577a6f46d642d4b81805ede9872490, scope.project.id=3325fd0bf7e2439aa626201e2ae25c6b, user.id=7cf4904211ee414793d522ae570a016e) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:17:31.906053 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4919016-476f-4167-8ee2-85a34e845ff7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:31.908015 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4919016-476f-4167-8ee2-85a34e845ff7 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:31.914661 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 102/203] 10.222.0.102 () {56 vars in 1379 bytes} [Thu Sep 1 17:17:31 2022] GET /identity/v3/role_assignments?user.id=7cf4904211ee414793d522ae570a016e&scope.project.id=3325fd0bf7e2439aa626201e2ae25c6b&role.id=36577a6f46d642d4b81805ede9872490 => generated 260 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:31.915705 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4919016-476f-4167-8ee2-85a34e845ff7 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:31.992182 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6150a7-3220-44ae-94a6-366cedc42b53 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:32.015373 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:32.030336 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096d0c45-d88d-4b0f-b200-00fc683387ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:32.031097 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096d0c45-d88d-4b0f-b200-00fc683387ee None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:32.031386 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096d0c45-d88d-4b0f-b200-00fc683387ee None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:32.031728 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096d0c45-d88d-4b0f-b200-00fc683387ee None admin] PATH_INFO: `/v3/users/f8579d2130b94c37ad6f44b1a88786bf` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:32.032926 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096d0c45-d88d-4b0f-b200-00fc683387ee None admin] RBAC: Authorizing `identity:get_user(user_id=f8579d2130b94c37ad6f44b1a88786bf)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:32.034653 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096d0c45-d88d-4b0f-b200-00fc683387ee None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:32.036477 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 102/204] 10.222.0.102 () {56 vars in 1182 bytes} [Thu Sep 1 17:17:31 2022] GET /identity/v3/users/f8579d2130b94c37ad6f44b1a88786bf => generated 290 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:32.037404 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4919016-476f-4167-8ee2-85a34e845ff7 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:32.042776 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096d0c45-d88d-4b0f-b200-00fc683387ee None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:32.077782 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd90abe5-2a7a-4333-83a8-eb306393bb80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:32.078736 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd90abe5-2a7a-4333-83a8-eb306393bb80 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:32.079210 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd90abe5-2a7a-4333-83a8-eb306393bb80 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:32.079660 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd90abe5-2a7a-4333-83a8-eb306393bb80 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:32.080673 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd90abe5-2a7a-4333-83a8-eb306393bb80 None admin] RBAC: Adding query filter params (role.id=abef78c185c74c16a2c7e8a170583f9f, scope.project.id=3325fd0bf7e2439aa626201e2ae25c6b, user.id=7cf4904211ee414793d522ae570a016e) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:17:32.081162 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd90abe5-2a7a-4333-83a8-eb306393bb80 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:32.083489 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd90abe5-2a7a-4333-83a8-eb306393bb80 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:32.090148 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 103/205] 10.222.0.102 () {56 vars in 1379 bytes} [Thu Sep 1 17:17:31 2022] GET /identity/v3/role_assignments?user.id=7cf4904211ee414793d522ae570a016e&scope.project.id=3325fd0bf7e2439aa626201e2ae25c6b&role.id=abef78c185c74c16a2c7e8a170583f9f => generated 260 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:32.097911 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:32.161175 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096d0c45-d88d-4b0f-b200-00fc683387ee None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:32.203364 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565f244c-5d71-414e-a680-9df70802b050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:32.203997 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565f244c-5d71-414e-a680-9df70802b050 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:32.204275 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565f244c-5d71-414e-a680-9df70802b050 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:32.204544 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565f244c-5d71-414e-a680-9df70802b050 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:32.280499 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565f244c-5d71-414e-a680-9df70802b050 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:32.281993 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565f244c-5d71-414e-a680-9df70802b050 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:32.325774 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:32.387712 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 103/206] 10.222.0.102 () {58 vars in 1287 bytes} [Thu Sep 1 17:17:32 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 17:17:32.393546 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd90abe5-2a7a-4333-83a8-eb306393bb80 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:32.518376 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd90abe5-2a7a-4333-83a8-eb306393bb80 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:32.560962 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39127557-842f-4e91-9172-84392a863f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:32.561619 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39127557-842f-4e91-9172-84392a863f5a None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:32.561913 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39127557-842f-4e91-9172-84392a863f5a None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:32.562191 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39127557-842f-4e91-9172-84392a863f5a None admin] PATH_INFO: `/v3/projects/8e88c67b942f45108c95a833547f9bc8` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:32.563349 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39127557-842f-4e91-9172-84392a863f5a None admin] RBAC: Authorizing `identity:get_project(project_id=8e88c67b942f45108c95a833547f9bc8)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:32.564914 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39127557-842f-4e91-9172-84392a863f5a None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:32.571157 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 104/207] 10.222.0.102 () {56 vars in 1191 bytes} [Thu Sep 1 17:17:32 2022] GET /identity/v3/projects/8e88c67b942f45108c95a833547f9bc8 => generated 299 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:32.577404 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565f244c-5d71-414e-a680-9df70802b050 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:32.736129 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565f244c-5d71-414e-a680-9df70802b050 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:32.787137 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f16e5c-caaf-4bbc-b776-c693298b5684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:32.787745 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f16e5c-caaf-4bbc-b776-c693298b5684 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:32.788020 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f16e5c-caaf-4bbc-b776-c693298b5684 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:32.788286 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f16e5c-caaf-4bbc-b776-c693298b5684 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:32.789036 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f16e5c-caaf-4bbc-b776-c693298b5684 None admin] RBAC: Adding query filter params (role.id=ab816d03be024513884447335f2de35a, scope.project.id=8e88c67b942f45108c95a833547f9bc8, user.id=f8579d2130b94c37ad6f44b1a88786bf) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:17:32.789428 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f16e5c-caaf-4bbc-b776-c693298b5684 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:32.790849 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f16e5c-caaf-4bbc-b776-c693298b5684 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:32.801453 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 104/208] 10.222.0.102 () {56 vars in 1379 bytes} [Thu Sep 1 17:17:32 2022] GET /identity/v3/role_assignments?user.id=f8579d2130b94c37ad6f44b1a88786bf&scope.project.id=8e88c67b942f45108c95a833547f9bc8&role.id=ab816d03be024513884447335f2de35a => generated 610 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:33.245706 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e3d100-ca17-4ab1-a524-17913ccd72be None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:33.246045 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e3d100-ca17-4ab1-a524-17913ccd72be None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:33.246230 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e3d100-ca17-4ab1-a524-17913ccd72be None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:33.247681 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 105/209] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:17:33 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:17:33.253206 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675f3185-640e-49f3-826d-47fbde70a16d None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:33.253527 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675f3185-640e-49f3-826d-47fbde70a16d None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:33.253808 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675f3185-640e-49f3-826d-47fbde70a16d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:33.269809 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:33.276041 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-675f3185-640e-49f3-826d-47fbde70a16d None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:17:33.277808 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-675f3185-640e-49f3-826d-47fbde70a16d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:17:33.350679 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10a78c2-54dd-4766-8650-263142951541 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:33.350997 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10a78c2-54dd-4766-8650-263142951541 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:33.351240 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10a78c2-54dd-4766-8650-263142951541 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:33.353026 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 106/210] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:17:33 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Sep 01 17:17:33.362788 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d627ba12-2d8d-4412-8e7d-07ba44f3e196 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:33.363085 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d627ba12-2d8d-4412-8e7d-07ba44f3e196 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:33.363350 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d627ba12-2d8d-4412-8e7d-07ba44f3e196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:33.363738 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 105/211] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:17:33 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:17:33.382633 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-d627ba12-2d8d-4412-8e7d-07ba44f3e196 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:17:33.383827 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-d627ba12-2d8d-4412-8e7d-07ba44f3e196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:17:33.398813 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:33.454940 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 107/212] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:17:33 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 17:17:33.640106 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675f3185-640e-49f3-826d-47fbde70a16d None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:33.724013 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d627ba12-2d8d-4412-8e7d-07ba44f3e196 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:33.743277 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675f3185-640e-49f3-826d-47fbde70a16d None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:33.790976 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3630658e-39f2-4931-ba83-b54c5a71d492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:33.791618 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3630658e-39f2-4931-ba83-b54c5a71d492 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:33.791899 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3630658e-39f2-4931-ba83-b54c5a71d492 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:33.792172 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3630658e-39f2-4931-ba83-b54c5a71d492 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:33.795131 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:33.804613 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3630658e-39f2-4931-ba83-b54c5a71d492 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:17:33.809771 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-3630658e-39f2-4931-ba83-b54c5a71d492 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:17:33.809771 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:17:33.809771 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:17:33.809771 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:17:33.809771 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:17:33.809771 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:17:33.809771 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:17:33.809771 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:17:33.809771 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:17:33.809771 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:17:33.809771 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:17:33.809771 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:17:33.809771 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:17:33.809771 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:17:33.809771 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:17:33.809771 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:17:33.810634 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:17:33.810634 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:17:33.810634 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:17:33.810634 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:17:33.810634 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:17:33.810634 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:17:33.810634 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:17:33.810634 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:17:33.810634 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:17:33.810634 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:17:33.810634 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:17:33.810634 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:17:33.810634 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:17:33.810634 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:17:33.810634 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:17:33.810634 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:17:33.812014 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:17:33.812014 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:17:33.812014 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:17:33.812014 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:17:33.812014 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:17:33.812014 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:17:33.812014 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:17:33.812014 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:17:33.812014 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:17:33.812014 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:17:33.812014 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:17:33.812014 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:17:33.812014 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 106/213] 10.222.0.102 () {56 vars in 1104 bytes} [Thu Sep 1 17:17:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:17:33.822268 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3630658e-39f2-4931-ba83-b54c5a71d492 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:33.849404 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d627ba12-2d8d-4412-8e7d-07ba44f3e196 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:33.889247 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c199154b-f35b-47ff-a49d-6aaa6ebd3492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:33.890424 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c199154b-f35b-47ff-a49d-6aaa6ebd3492 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:33.890904 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c199154b-f35b-47ff-a49d-6aaa6ebd3492 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:33.891386 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c199154b-f35b-47ff-a49d-6aaa6ebd3492 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:33.892805 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c199154b-f35b-47ff-a49d-6aaa6ebd3492 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:33.894574 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c199154b-f35b-47ff-a49d-6aaa6ebd3492 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:33.896618 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 108/214] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:17:33 2022] GET /identity/v3/domains/default => generated 200 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:17:33.902403 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c199154b-f35b-47ff-a49d-6aaa6ebd3492 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:33.921801 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:33.938990 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3630658e-39f2-4931-ba83-b54c5a71d492 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:33.977849 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71fcb32-22e7-4606-a0aa-cb70b56b85bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:33.978544 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71fcb32-22e7-4606-a0aa-cb70b56b85bd None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:33.978822 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71fcb32-22e7-4606-a0aa-cb70b56b85bd None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:33.979151 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71fcb32-22e7-4606-a0aa-cb70b56b85bd None admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:33.979908 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71fcb32-22e7-4606-a0aa-cb70b56b85bd None admin] RBAC: Adding query filter params (name=member) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:17:33.980114 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71fcb32-22e7-4606-a0aa-cb70b56b85bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:33.981523 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71fcb32-22e7-4606-a0aa-cb70b56b85bd None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:33.990674 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 107/215] 10.222.0.102 () {56 vars in 1106 bytes} [Thu Sep 1 17:17:33 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:33.996630 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71fcb32-22e7-4606-a0aa-cb70b56b85bd None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:34.015661 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c199154b-f35b-47ff-a49d-6aaa6ebd3492 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:34.054336 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc428e64-59cb-4aac-b999-74ce842d624a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:34.055301 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc428e64-59cb-4aac-b999-74ce842d624a None admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:34.055776 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc428e64-59cb-4aac-b999-74ce842d624a None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:34.056246 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc428e64-59cb-4aac-b999-74ce842d624a None admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:34.057237 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc428e64-59cb-4aac-b999-74ce842d624a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:34.059101 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc428e64-59cb-4aac-b999-74ce842d624a None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:34.060516 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:34.093352 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 109/216] 10.222.0.102 () {60 vars in 1137 bytes} [Thu Sep 1 17:17:33 2022] POST /identity/v3/users => generated 298 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:17:34.125162 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71fcb32-22e7-4606-a0aa-cb70b56b85bd None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:34.166021 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581723c4-35de-441a-b0d0-fe4c1ec28126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:34.166582 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581723c4-35de-441a-b0d0-fe4c1ec28126 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:34.166862 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581723c4-35de-441a-b0d0-fe4c1ec28126 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:34.167133 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581723c4-35de-441a-b0d0-fe4c1ec28126 None admin] PATH_INFO: `/v3/users/0dc9f9b26c9b4337b4ecba20f6cecf5e` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:34.185386 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581723c4-35de-441a-b0d0-fe4c1ec28126 None admin] RBAC: Authorizing `identity:get_user(user_id=0dc9f9b26c9b4337b4ecba20f6cecf5e)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:34.186924 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581723c4-35de-441a-b0d0-fe4c1ec28126 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:34.188644 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 108/217] 10.222.0.102 () {56 vars in 1182 bytes} [Thu Sep 1 17:17:33 2022] GET /identity/v3/users/0dc9f9b26c9b4337b4ecba20f6cecf5e => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:17:34.195398 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc428e64-59cb-4aac-b999-74ce842d624a None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:34.230839 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eff457-8ec0-4393-a20f-a269a0c13850 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:34.231169 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eff457-8ec0-4393-a20f-a269a0c13850 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:34.231452 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eff457-8ec0-4393-a20f-a269a0c13850 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:34.233100 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 109/218] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:17:34 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:17:34.238946 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a5f093-4e2a-4d78-b113-4c179aa67d28 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:34.239382 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a5f093-4e2a-4d78-b113-4c179aa67d28 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:34.239753 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a5f093-4e2a-4d78-b113-4c179aa67d28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:34.260179 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-59a5f093-4e2a-4d78-b113-4c179aa67d28 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:17:34.261737 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-59a5f093-4e2a-4d78-b113-4c179aa67d28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:17:34.285124 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:34.321328 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc428e64-59cb-4aac-b999-74ce842d624a None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:34.347401 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 110/219] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:17:34 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 17:17:34.364740 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1649b633-93a7-4eb8-9be9-a2fcc1907129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:34.365705 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1649b633-93a7-4eb8-9be9-a2fcc1907129 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:34.366165 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1649b633-93a7-4eb8-9be9-a2fcc1907129 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:34.366612 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1649b633-93a7-4eb8-9be9-a2fcc1907129 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:34.439028 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1649b633-93a7-4eb8-9be9-a2fcc1907129 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:34.440713 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1649b633-93a7-4eb8-9be9-a2fcc1907129 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:34.475692 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dbe8f6-de0b-43e2-93f2-af3cdc17f8a4 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:34.480274 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dbe8f6-de0b-43e2-93f2-af3cdc17f8a4 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:34.480807 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dbe8f6-de0b-43e2-93f2-af3cdc17f8a4 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:34.482486 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 111/220] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:17:34 2022] GET /identity => generated 271 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Sep 01 17:17:34.487762 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd28ee63-ded1-46de-a5a6-cbc9e0872dcc None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:34.488296 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd28ee63-ded1-46de-a5a6-cbc9e0872dcc None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:34.489334 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd28ee63-ded1-46de-a5a6-cbc9e0872dcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:34.511662 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-bd28ee63-ded1-46de-a5a6-cbc9e0872dcc None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:17:34.513217 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-bd28ee63-ded1-46de-a5a6-cbc9e0872dcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:17:34.521255 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:34.563094 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 110/221] 10.222.0.102 () {58 vars in 1287 bytes} [Thu Sep 1 17:17:34 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 17:17:34.569766 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1649b633-93a7-4eb8-9be9-a2fcc1907129 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:34.577722 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:34.612480 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 112/222] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:17:34 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:17:34.644159 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd28ee63-ded1-46de-a5a6-cbc9e0872dcc None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:34.687540 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1649b633-93a7-4eb8-9be9-a2fcc1907129 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:34.724894 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8471e70e-4be3-442d-a30b-5c6c07d85484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:34.726262 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8471e70e-4be3-442d-a30b-5c6c07d85484 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:34.726690 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8471e70e-4be3-442d-a30b-5c6c07d85484 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:34.727388 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8471e70e-4be3-442d-a30b-5c6c07d85484 None admin] PATH_INFO: `/v3/projects/3325fd0bf7e2439aa626201e2ae25c6b` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:34.728699 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8471e70e-4be3-442d-a30b-5c6c07d85484 None admin] RBAC: Authorizing `identity:get_project(project_id=3325fd0bf7e2439aa626201e2ae25c6b)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:34.730396 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8471e70e-4be3-442d-a30b-5c6c07d85484 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:34.732273 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 111/223] 10.222.0.102 () {56 vars in 1191 bytes} [Thu Sep 1 17:17:34 2022] GET /identity/v3/projects/3325fd0bf7e2439aa626201e2ae25c6b => generated 303 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:34.742231 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8471e70e-4be3-442d-a30b-5c6c07d85484 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:34.783899 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd28ee63-ded1-46de-a5a6-cbc9e0872dcc None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:34.820548 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:34.829090 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5882a1-f91c-4085-a39a-ecaa29901bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:34.829761 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5882a1-f91c-4085-a39a-ecaa29901bf7 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:34.830039 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5882a1-f91c-4085-a39a-ecaa29901bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:34.830308 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5882a1-f91c-4085-a39a-ecaa29901bf7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:34.855920 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5882a1-f91c-4085-a39a-ecaa29901bf7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:17:34.861228 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-ff5882a1-f91c-4085-a39a-ecaa29901bf7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:17:34.861228 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:17:34.861228 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:17:34.861228 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:17:34.861228 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:17:34.861228 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:17:34.861228 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:17:34.861228 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:17:34.861228 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:17:34.861228 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:17:34.861228 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:17:34.861228 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:17:34.861228 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:17:34.861228 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:17:34.861228 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:17:34.861228 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:17:34.862085 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:17:34.862085 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:17:34.862085 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:17:34.862085 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:17:34.862085 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:17:34.862085 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:17:34.862085 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:17:34.862085 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:17:34.862085 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:17:34.862085 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:17:34.862085 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:17:34.862085 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:17:34.862085 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:17:34.862085 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:17:34.862085 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:17:34.862085 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:17:34.862872 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:17:34.862872 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:17:34.862872 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:17:34.862872 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:17:34.862872 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:17:34.862872 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:17:34.862872 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:17:34.862872 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:17:34.862872 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:17:34.862872 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:17:34.862872 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:17:34.862872 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:17:34.865527 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 113/224] 10.222.0.102 () {56 vars in 1101 bytes} [Thu Sep 1 17:17:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:17:34.874269 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5882a1-f91c-4085-a39a-ecaa29901bf7 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:34.934173 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8471e70e-4be3-442d-a30b-5c6c07d85484 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:34.949691 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:34.998133 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b18a91-6045-4da0-89ab-eab5cf755473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:34.998664 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b18a91-6045-4da0-89ab-eab5cf755473 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:34.998947 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b18a91-6045-4da0-89ab-eab5cf755473 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:34.999218 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b18a91-6045-4da0-89ab-eab5cf755473 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:34.999945 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b18a91-6045-4da0-89ab-eab5cf755473 None admin] RBAC: Adding query filter params (role.id=b75cb360925e4ba5a65f14965c103ffd, scope.project.id=3325fd0bf7e2439aa626201e2ae25c6b, user.id=0dc9f9b26c9b4337b4ecba20f6cecf5e) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:17:35.000253 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b18a91-6045-4da0-89ab-eab5cf755473 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:35.001721 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b18a91-6045-4da0-89ab-eab5cf755473 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:35.007782 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 112/225] 10.222.0.102 () {56 vars in 1379 bytes} [Thu Sep 1 17:17:34 2022] GET /identity/v3/role_assignments?user.id=0dc9f9b26c9b4337b4ecba20f6cecf5e&scope.project.id=3325fd0bf7e2439aa626201e2ae25c6b&role.id=b75cb360925e4ba5a65f14965c103ffd => generated 260 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:35.008736 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b18a91-6045-4da0-89ab-eab5cf755473 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:35.077106 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5882a1-f91c-4085-a39a-ecaa29901bf7 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:35.163274 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db0af70-7462-4b74-b555-621889a85d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:35.163900 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db0af70-7462-4b74-b555-621889a85d39 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:35.164295 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db0af70-7462-4b74-b555-621889a85d39 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:35.164594 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db0af70-7462-4b74-b555-621889a85d39 None admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:35.165269 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db0af70-7462-4b74-b555-621889a85d39 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:17:35.165592 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db0af70-7462-4b74-b555-621889a85d39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:35.166968 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db0af70-7462-4b74-b555-621889a85d39 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:35.184944 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 114/226] 10.222.0.102 () {56 vars in 1104 bytes} [Thu Sep 1 17:17:34 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:35.186366 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db0af70-7462-4b74-b555-621889a85d39 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:35.195757 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b18a91-6045-4da0-89ab-eab5cf755473 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:35.198230 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:35.238125 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bd2d8e-7aa2-48ef-a25d-305f41ccec7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:35.239342 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bd2d8e-7aa2-48ef-a25d-305f41ccec7b None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:35.239621 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bd2d8e-7aa2-48ef-a25d-305f41ccec7b None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:35.239888 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bd2d8e-7aa2-48ef-a25d-305f41ccec7b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:35.260361 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bd2d8e-7aa2-48ef-a25d-305f41ccec7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:17:35.269232 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-f5bd2d8e-7aa2-48ef-a25d-305f41ccec7b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 17:17:35.269232 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:17:35.269232 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:17:35.269232 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:17:35.269232 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:17:35.269232 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:17:35.269232 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:17:35.269232 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:17:35.269232 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:17:35.269232 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:17:35.269232 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:17:35.269232 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:17:35.269232 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:17:35.269232 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:17:35.269232 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:17:35.269232 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:17:35.270187 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:17:35.270187 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:17:35.270187 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:17:35.270187 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:17:35.270187 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:17:35.270187 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:17:35.270187 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:17:35.270187 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:17:35.270187 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:17:35.270187 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:17:35.270187 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:17:35.270187 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:17:35.270187 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:17:35.270187 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:17:35.270187 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:17:35.270187 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:17:35.270964 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:17:35.270964 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:17:35.270964 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:17:35.270964 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:17:35.270964 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:17:35.270964 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:17:35.270964 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:17:35.270964 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:17:35.270964 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:17:35.270964 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:17:35.270964 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 17:17:35.270964 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:17:35.270964 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 113/227] 10.222.0.102 () {56 vars in 1119 bytes} [Thu Sep 1 17:17:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:17:35.277560 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bd2d8e-7aa2-48ef-a25d-305f41ccec7b None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:35.357685 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:35.362919 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db0af70-7462-4b74-b555-621889a85d39 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:35.432720 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3447451-c756-41d3-80d7-c544b6877294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:35.433387 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3447451-c756-41d3-80d7-c544b6877294 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:35.433671 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3447451-c756-41d3-80d7-c544b6877294 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:35.433795 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bd2d8e-7aa2-48ef-a25d-305f41ccec7b None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:35.433969 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3447451-c756-41d3-80d7-c544b6877294 None admin] PATH_INFO: `/v3/users/7cf4904211ee414793d522ae570a016e` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:35.435090 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3447451-c756-41d3-80d7-c544b6877294 None admin] RBAC: Authorizing `identity:get_user(user_id=7cf4904211ee414793d522ae570a016e)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:35.436533 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3447451-c756-41d3-80d7-c544b6877294 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:35.446249 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 115/228] 10.222.0.102 () {56 vars in 1182 bytes} [Thu Sep 1 17:17:35 2022] GET /identity/v3/users/7cf4904211ee414793d522ae570a016e => generated 284 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:35.452003 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3447451-c756-41d3-80d7-c544b6877294 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:35.469232 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19b2ce3-bfc6-458c-97b4-755f538f7c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:35.469868 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19b2ce3-bfc6-458c-97b4-755f538f7c32 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:35.470143 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19b2ce3-bfc6-458c-97b4-755f538f7c32 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:35.470419 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19b2ce3-bfc6-458c-97b4-755f538f7c32 None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:35.471136 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19b2ce3-bfc6-458c-97b4-755f538f7c32 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:17:35.471436 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19b2ce3-bfc6-458c-97b4-755f538f7c32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:35.472873 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19b2ce3-bfc6-458c-97b4-755f538f7c32 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:35.482217 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 114/229] 10.222.0.102 () {56 vars in 1116 bytes} [Thu Sep 1 17:17:35 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:35.487570 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19b2ce3-bfc6-458c-97b4-755f538f7c32 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:35.549762 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:35.626504 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19b2ce3-bfc6-458c-97b4-755f538f7c32 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:35.647447 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3447451-c756-41d3-80d7-c544b6877294 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:35.660316 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd930eb1-044b-4a88-aaae-62bf1c64dd08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:35.661523 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd930eb1-044b-4a88-aaae-62bf1c64dd08 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:35.661801 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd930eb1-044b-4a88-aaae-62bf1c64dd08 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:35.662068 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd930eb1-044b-4a88-aaae-62bf1c64dd08 None admin] PATH_INFO: `/v3/users/7cf4904211ee414793d522ae570a016e` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:35.663195 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd930eb1-044b-4a88-aaae-62bf1c64dd08 None admin] RBAC: Authorizing `identity:get_user(user_id=7cf4904211ee414793d522ae570a016e)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:35.669346 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd930eb1-044b-4a88-aaae-62bf1c64dd08 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:35.669346 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 115/230] 10.222.0.102 () {56 vars in 1182 bytes} [Thu Sep 1 17:17:35 2022] GET /identity/v3/users/7cf4904211ee414793d522ae570a016e => generated 284 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:35.671904 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd930eb1-044b-4a88-aaae-62bf1c64dd08 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:35.726069 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d520f3-3dc1-458c-a2be-b6e89e34f5a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:35.727523 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d520f3-3dc1-458c-a2be-b6e89e34f5a3 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:35.728065 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d520f3-3dc1-458c-a2be-b6e89e34f5a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:35.728548 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d520f3-3dc1-458c-a2be-b6e89e34f5a3 None admin] PATH_INFO: `/v3/projects/3325fd0bf7e2439aa626201e2ae25c6b` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:35.730257 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d520f3-3dc1-458c-a2be-b6e89e34f5a3 None admin] RBAC: Authorizing `identity:get_project(project_id=3325fd0bf7e2439aa626201e2ae25c6b)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:35.732032 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d520f3-3dc1-458c-a2be-b6e89e34f5a3 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:35.738026 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 116/231] 10.222.0.102 () {56 vars in 1191 bytes} [Thu Sep 1 17:17:35 2022] GET /identity/v3/projects/3325fd0bf7e2439aa626201e2ae25c6b => generated 303 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:17:35.745402 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d520f3-3dc1-458c-a2be-b6e89e34f5a3 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:35.775027 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd930eb1-044b-4a88-aaae-62bf1c64dd08 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:35.786241 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:35.829659 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:35.843672 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e319391-a318-4fb3-9504-aae24fd918b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:35.844310 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e319391-a318-4fb3-9504-aae24fd918b1 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:35.844607 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e319391-a318-4fb3-9504-aae24fd918b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:35.844852 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e319391-a318-4fb3-9504-aae24fd918b1 None admin] PATH_INFO: `/v3/projects/3325fd0bf7e2439aa626201e2ae25c6b` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:35.846104 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e319391-a318-4fb3-9504-aae24fd918b1 None admin] RBAC: Authorizing `identity:get_project(project_id=3325fd0bf7e2439aa626201e2ae25c6b)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:35.851354 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e319391-a318-4fb3-9504-aae24fd918b1 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:35.853575 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 116/232] 10.222.0.102 () {56 vars in 1191 bytes} [Thu Sep 1 17:17:35 2022] GET /identity/v3/projects/3325fd0bf7e2439aa626201e2ae25c6b => generated 303 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:17:35.864254 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e319391-a318-4fb3-9504-aae24fd918b1 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:36.047311 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d520f3-3dc1-458c-a2be-b6e89e34f5a3 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:36.079725 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e319391-a318-4fb3-9504-aae24fd918b1 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:36.131543 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8efd49-3e0a-4097-bb32-ff4fa4c844ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:36.132211 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8efd49-3e0a-4097-bb32-ff4fa4c844ce None admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:36.132488 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8efd49-3e0a-4097-bb32-ff4fa4c844ce None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:36.132758 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8efd49-3e0a-4097-bb32-ff4fa4c844ce None admin] PATH_INFO: `/v3/projects/3325fd0bf7e2439aa626201e2ae25c6b/users/7cf4904211ee414793d522ae570a016e/roles/36577a6f46d642d4b81805ede9872490` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:36.136781 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8efd49-3e0a-4097-bb32-ff4fa4c844ce None admin] RBAC: Authorizing `identity:create_grant(project_id=3325fd0bf7e2439aa626201e2ae25c6b, user_id=7cf4904211ee414793d522ae570a016e, role_id=36577a6f46d642d4b81805ede9872490)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:36.138173 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8efd49-3e0a-4097-bb32-ff4fa4c844ce None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:36.161333 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39443871-3640-4ba3-9132-29b58819bb4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:36.161333 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39443871-3640-4ba3-9132-29b58819bb4d None admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:36.161333 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39443871-3640-4ba3-9132-29b58819bb4d None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:36.161333 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39443871-3640-4ba3-9132-29b58819bb4d None admin] PATH_INFO: `/v3/projects/3325fd0bf7e2439aa626201e2ae25c6b/users/7cf4904211ee414793d522ae570a016e/roles/abef78c185c74c16a2c7e8a170583f9f` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:36.173118 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39443871-3640-4ba3-9132-29b58819bb4d None admin] RBAC: Authorizing `identity:create_grant(project_id=3325fd0bf7e2439aa626201e2ae25c6b, user_id=7cf4904211ee414793d522ae570a016e, role_id=abef78c185c74c16a2c7e8a170583f9f)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:36.184319 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 117/233] 10.222.0.102 () {58 vars in 1444 bytes} [Thu Sep 1 17:17:35 2022] PUT /identity/v3/projects/3325fd0bf7e2439aa626201e2ae25c6b/users/7cf4904211ee414793d522ae570a016e/roles/36577a6f46d642d4b81805ede9872490 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:17:36.184808 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39443871-3640-4ba3-9132-29b58819bb4d None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:36.215476 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 117/234] 10.222.0.102 () {58 vars in 1444 bytes} [Thu Sep 1 17:17:35 2022] PUT /identity/v3/projects/3325fd0bf7e2439aa626201e2ae25c6b/users/7cf4904211ee414793d522ae570a016e/roles/abef78c185c74c16a2c7e8a170583f9f => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:17:36.649561 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495afc67-b594-4db2-9a35-9500fa30f2a7 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:36.650070 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495afc67-b594-4db2-9a35-9500fa30f2a7 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:36.650144 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495afc67-b594-4db2-9a35-9500fa30f2a7 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:36.651775 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 118/235] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:17:36 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:17:36.657007 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedd30ec-3833-4c38-b800-1a02f6790bf8 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:36.657570 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedd30ec-3833-4c38-b800-1a02f6790bf8 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:36.658048 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedd30ec-3833-4c38-b800-1a02f6790bf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:36.686412 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-bedd30ec-3833-4c38-b800-1a02f6790bf8 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:17:36.687954 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-bedd30ec-3833-4c38-b800-1a02f6790bf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:17:36.706721 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be84edb5-41d4-4fc1-8e4e-77e3a71619a0 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:36.707334 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be84edb5-41d4-4fc1-8e4e-77e3a71619a0 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:36.707836 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be84edb5-41d4-4fc1-8e4e-77e3a71619a0 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:36.709664 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 119/236] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:17:36 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:17:36.711473 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:36.715372 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d736e583-cec0-440c-9d4b-662d8dfb23f6 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:36.715901 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d736e583-cec0-440c-9d4b-662d8dfb23f6 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:36.716373 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d736e583-cec0-440c-9d4b-662d8dfb23f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:36.741171 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-d736e583-cec0-440c-9d4b-662d8dfb23f6 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:17:36.744837 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-d736e583-cec0-440c-9d4b-662d8dfb23f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:17:36.825854 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 118/237] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:17:36 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:17:36.894647 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 120/238] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:17:36 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 17:17:37.106150 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedd30ec-3833-4c38-b800-1a02f6790bf8 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:37.230312 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedd30ec-3833-4c38-b800-1a02f6790bf8 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:37.267509 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882afd18-353c-4437-bf9c-5bed4af40c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:37.268539 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882afd18-353c-4437-bf9c-5bed4af40c87 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:37.269015 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882afd18-353c-4437-bf9c-5bed4af40c87 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:37.269672 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882afd18-353c-4437-bf9c-5bed4af40c87 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:37.280480 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882afd18-353c-4437-bf9c-5bed4af40c87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:17:37.286164 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-882afd18-353c-4437-bf9c-5bed4af40c87 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 17:17:37.286164 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:17:37.286164 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:17:37.286164 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:17:37.286164 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:17:37.286164 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:17:37.286164 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:17:37.286164 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:17:37.286164 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:17:37.286164 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:17:37.286164 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:17:37.286164 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:17:37.286164 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:17:37.286164 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:17:37.286164 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:17:37.286164 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:17:37.287034 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:17:37.287034 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:17:37.287034 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:17:37.287034 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:17:37.287034 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:17:37.287034 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:17:37.287034 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:17:37.287034 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:17:37.287034 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:17:37.287034 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:17:37.287034 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:17:37.287034 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:17:37.287034 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:17:37.287034 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:17:37.287034 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:17:37.287034 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:17:37.287817 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:17:37.287817 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:17:37.287817 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:17:37.287817 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:17:37.287817 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:17:37.287817 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:17:37.287817 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:17:37.287817 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:17:37.287817 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:17:37.287817 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:17:37.287817 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 17:17:37.287817 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:17:37.294943 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 119/239] 10.222.0.102 () {56 vars in 1104 bytes} [Thu Sep 1 17:17:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:17:37.296565 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d736e583-cec0-440c-9d4b-662d8dfb23f6 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:37.316667 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882afd18-353c-4437-bf9c-5bed4af40c87 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:37.333982 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:37.340230 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:37.417174 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d736e583-cec0-440c-9d4b-662d8dfb23f6 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:37.456252 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72dd3a9-36aa-4794-881b-a22ac3407657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:37.457212 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72dd3a9-36aa-4794-881b-a22ac3407657 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:37.457737 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72dd3a9-36aa-4794-881b-a22ac3407657 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:37.458242 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72dd3a9-36aa-4794-881b-a22ac3407657 None admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:37.459110 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72dd3a9-36aa-4794-881b-a22ac3407657 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:17:37.459641 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72dd3a9-36aa-4794-881b-a22ac3407657 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:37.461353 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72dd3a9-36aa-4794-881b-a22ac3407657 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:37.475349 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 121/240] 10.222.0.102 () {56 vars in 1106 bytes} [Thu Sep 1 17:17:37 2022] GET /identity/v3/roles?name=reader => generated 343 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:17:37.477615 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72dd3a9-36aa-4794-881b-a22ac3407657 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:37.558342 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882afd18-353c-4437-bf9c-5bed4af40c87 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:37.595908 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cf9935-d2b1-4d09-96e9-8064b1aa74d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:37.596803 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cf9935-d2b1-4d09-96e9-8064b1aa74d0 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:37.597222 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cf9935-d2b1-4d09-96e9-8064b1aa74d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:37.597755 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cf9935-d2b1-4d09-96e9-8064b1aa74d0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:37.599166 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cf9935-d2b1-4d09-96e9-8064b1aa74d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:37.601184 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cf9935-d2b1-4d09-96e9-8064b1aa74d0 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:37.603688 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 120/241] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:17:37 2022] GET /identity/v3/domains/default => generated 200 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:17:37.605092 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e84210-1bc3-4434-9338-5ca28cfa6865 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:37.605598 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e84210-1bc3-4434-9338-5ca28cfa6865 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:37.606027 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e84210-1bc3-4434-9338-5ca28cfa6865 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:37.607371 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 121/242] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:17:37 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:17:37.608603 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e84210-1bc3-4434-9338-5ca28cfa6865 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:37.626073 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:37.638453 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72dd3a9-36aa-4794-881b-a22ac3407657 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:37.685428 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb89fd8c-53ff-4880-a3e9-9487e09c3278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:37.686047 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb89fd8c-53ff-4880-a3e9-9487e09c3278 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:37.686259 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb89fd8c-53ff-4880-a3e9-9487e09c3278 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:37.686697 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb89fd8c-53ff-4880-a3e9-9487e09c3278 None admin] PATH_INFO: `/v3/users/ef6bf9831b334481b89ffb2c3d06fdce` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:37.707021 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:37.710088 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb89fd8c-53ff-4880-a3e9-9487e09c3278 None admin] RBAC: Authorizing `identity:get_user(user_id=ef6bf9831b334481b89ffb2c3d06fdce)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:37.713257 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb89fd8c-53ff-4880-a3e9-9487e09c3278 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:37.715937 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 122/243] 10.222.0.102 () {56 vars in 1182 bytes} [Thu Sep 1 17:17:37 2022] GET /identity/v3/users/ef6bf9831b334481b89ffb2c3d06fdce => generated 298 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:37.717936 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c42981-33e4-4db5-ab53-fa1a041ae4d9 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:37.718365 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c42981-33e4-4db5-ab53-fa1a041ae4d9 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:37.719087 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c42981-33e4-4db5-ab53-fa1a041ae4d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:37.731148 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e84210-1bc3-4434-9338-5ca28cfa6865 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:37.747287 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-76c42981-33e4-4db5-ab53-fa1a041ae4d9 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:17:37.748672 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-76c42981-33e4-4db5-ab53-fa1a041ae4d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:17:37.771777 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abf587a-60bb-46b9-88f0-96d293cd49a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:37.772519 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abf587a-60bb-46b9-88f0-96d293cd49a9 None admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:37.772909 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abf587a-60bb-46b9-88f0-96d293cd49a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:37.773274 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abf587a-60bb-46b9-88f0-96d293cd49a9 None admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:37.774287 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abf587a-60bb-46b9-88f0-96d293cd49a9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:37.775958 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abf587a-60bb-46b9-88f0-96d293cd49a9 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:37.799983 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 122/244] 10.222.0.102 () {60 vars in 1137 bytes} [Thu Sep 1 17:17:37 2022] POST /identity/v3/users => generated 294 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:17:37.801830 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abf587a-60bb-46b9-88f0-96d293cd49a9 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:37.823728 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 123/245] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:17:37 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:17:37.917107 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abf587a-60bb-46b9-88f0-96d293cd49a9 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:37.956593 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07f9c83-44b0-408b-bc45-87562653bdc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:37.957141 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07f9c83-44b0-408b-bc45-87562653bdc1 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:37.957484 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07f9c83-44b0-408b-bc45-87562653bdc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:37.957759 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07f9c83-44b0-408b-bc45-87562653bdc1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:37.972066 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:38.038489 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07f9c83-44b0-408b-bc45-87562653bdc1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:38.040065 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07f9c83-44b0-408b-bc45-87562653bdc1 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:38.126641 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c42981-33e4-4db5-ab53-fa1a041ae4d9 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:38.144370 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 123/246] 10.222.0.102 () {58 vars in 1287 bytes} [Thu Sep 1 17:17:37 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 17:17:38.152698 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07f9c83-44b0-408b-bc45-87562653bdc1 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:38.197933 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:38.257657 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:38.293733 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07f9c83-44b0-408b-bc45-87562653bdc1 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:38.294868 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c42981-33e4-4db5-ab53-fa1a041ae4d9 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:38.340317 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d2699e-4967-4b8f-8f95-ffcfc8860b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:38.341448 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d2699e-4967-4b8f-8f95-ffcfc8860b31 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:38.342018 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d2699e-4967-4b8f-8f95-ffcfc8860b31 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:38.342581 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d2699e-4967-4b8f-8f95-ffcfc8860b31 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:38.345099 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be55a2c9-0448-4c4a-bf8c-840d45d2cc8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:38.345836 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be55a2c9-0448-4c4a-bf8c-840d45d2cc8b None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:38.346119 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be55a2c9-0448-4c4a-bf8c-840d45d2cc8b None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:38.346390 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be55a2c9-0448-4c4a-bf8c-840d45d2cc8b None admin] PATH_INFO: `/v3/projects/3325fd0bf7e2439aa626201e2ae25c6b` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:38.351003 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be55a2c9-0448-4c4a-bf8c-840d45d2cc8b None admin] RBAC: Authorizing `identity:get_project(project_id=3325fd0bf7e2439aa626201e2ae25c6b)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:38.352971 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be55a2c9-0448-4c4a-bf8c-840d45d2cc8b None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:38.355240 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 124/247] 10.222.0.102 () {56 vars in 1191 bytes} [Thu Sep 1 17:17:38 2022] GET /identity/v3/projects/3325fd0bf7e2439aa626201e2ae25c6b => generated 303 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:38.357272 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d2699e-4967-4b8f-8f95-ffcfc8860b31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:17:38.360877 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be55a2c9-0448-4c4a-bf8c-840d45d2cc8b None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:38.362135 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-f9d2699e-4967-4b8f-8f95-ffcfc8860b31 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:17:38.362135 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:17:38.362135 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:17:38.362135 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:17:38.362135 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:17:38.362135 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:17:38.362135 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:17:38.362135 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:17:38.362135 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:17:38.362135 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:17:38.362135 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:17:38.362135 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:17:38.362135 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:17:38.362135 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:17:38.362135 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:17:38.362135 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:17:38.362979 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:17:38.362979 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:17:38.362979 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:17:38.362979 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:17:38.362979 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:17:38.362979 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:17:38.362979 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:17:38.362979 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:17:38.362979 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:17:38.362979 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:17:38.362979 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:17:38.362979 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:17:38.362979 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:17:38.362979 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:17:38.362979 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:17:38.362979 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:17:38.363819 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:17:38.363819 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:17:38.363819 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:17:38.363819 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:17:38.363819 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:17:38.363819 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:17:38.363819 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:17:38.363819 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:17:38.363819 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:17:38.363819 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:17:38.363819 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:17:38.363819 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:17:38.363819 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 124/248] 10.222.0.102 () {56 vars in 1104 bytes} [Thu Sep 1 17:17:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:17:38.369552 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d2699e-4967-4b8f-8f95-ffcfc8860b31 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:38.499044 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d2699e-4967-4b8f-8f95-ffcfc8860b31 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:38.504451 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be55a2c9-0448-4c4a-bf8c-840d45d2cc8b None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:38.507142 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:38.539675 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb29df4-3a24-4340-bb10-16f25e2e1246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:38.540217 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb29df4-3a24-4340-bb10-16f25e2e1246 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:38.540496 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb29df4-3a24-4340-bb10-16f25e2e1246 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:38.540940 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb29df4-3a24-4340-bb10-16f25e2e1246 None admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:38.542032 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb29df4-3a24-4340-bb10-16f25e2e1246 None admin] RBAC: Adding query filter params (name=member) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:17:38.542308 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb29df4-3a24-4340-bb10-16f25e2e1246 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:38.544042 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb29df4-3a24-4340-bb10-16f25e2e1246 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:38.545745 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:38.553537 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfbe462-2b57-4540-9a6f-375f64aeeed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:38.554154 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfbe462-2b57-4540-9a6f-375f64aeeed0 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:38.554430 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfbe462-2b57-4540-9a6f-375f64aeeed0 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:38.554707 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfbe462-2b57-4540-9a6f-375f64aeeed0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:38.555410 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 125/249] 10.222.0.102 () {56 vars in 1106 bytes} [Thu Sep 1 17:17:38 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:38.555529 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfbe462-2b57-4540-9a6f-375f64aeeed0 None admin] RBAC: Adding query filter params (role.id=ab816d03be024513884447335f2de35a, scope.project.id=3325fd0bf7e2439aa626201e2ae25c6b, user.id=ef6bf9831b334481b89ffb2c3d06fdce) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:17:38.555776 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfbe462-2b57-4540-9a6f-375f64aeeed0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:38.557331 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfbe462-2b57-4540-9a6f-375f64aeeed0 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:38.560867 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb29df4-3a24-4340-bb10-16f25e2e1246 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:38.565353 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 125/250] 10.222.0.102 () {56 vars in 1379 bytes} [Thu Sep 1 17:17:38 2022] GET /identity/v3/role_assignments?user.id=ef6bf9831b334481b89ffb2c3d06fdce&scope.project.id=3325fd0bf7e2439aa626201e2ae25c6b&role.id=ab816d03be024513884447335f2de35a => generated 260 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:17:38.680977 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb29df4-3a24-4340-bb10-16f25e2e1246 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:38.718170 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21a4ea0-84e5-4344-a10d-ddc58cb31f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:38.718803 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21a4ea0-84e5-4344-a10d-ddc58cb31f3c None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:38.719085 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21a4ea0-84e5-4344-a10d-ddc58cb31f3c None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:38.719356 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21a4ea0-84e5-4344-a10d-ddc58cb31f3c None admin] PATH_INFO: `/v3/users/0dc9f9b26c9b4337b4ecba20f6cecf5e` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:38.720608 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21a4ea0-84e5-4344-a10d-ddc58cb31f3c None admin] RBAC: Authorizing `identity:get_user(user_id=0dc9f9b26c9b4337b4ecba20f6cecf5e)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:38.722109 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21a4ea0-84e5-4344-a10d-ddc58cb31f3c None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:38.724475 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 126/251] 10.222.0.102 () {56 vars in 1182 bytes} [Thu Sep 1 17:17:38 2022] GET /identity/v3/users/0dc9f9b26c9b4337b4ecba20f6cecf5e => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:17:38.730422 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfbe462-2b57-4540-9a6f-375f64aeeed0 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:38.767670 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6bd595-98f6-4505-a1c0-8a63d8d741b7 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:38.768042 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6bd595-98f6-4505-a1c0-8a63d8d741b7 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:38.768218 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6bd595-98f6-4505-a1c0-8a63d8d741b7 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:38.769955 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 127/252] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:17:38 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Sep 01 17:17:38.775218 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49917140-0e46-42fa-9dd8-1901c2d9cd51 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:38.775499 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49917140-0e46-42fa-9dd8-1901c2d9cd51 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:38.775773 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49917140-0e46-42fa-9dd8-1901c2d9cd51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:38.794652 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-49917140-0e46-42fa-9dd8-1901c2d9cd51 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:17:38.795760 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-49917140-0e46-42fa-9dd8-1901c2d9cd51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:17:38.844928 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfbe462-2b57-4540-9a6f-375f64aeeed0 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:38.865169 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:38.870922 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 128/253] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:17:38 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 17:17:38.880526 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a33b29-4c86-4a69-b5b7-237bf0ea4d7f None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:38.881051 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a33b29-4c86-4a69-b5b7-237bf0ea4d7f None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:38.881520 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a33b29-4c86-4a69-b5b7-237bf0ea4d7f None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:38.883474 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 129/254] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:17:38 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Sep 01 17:17:38.888719 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877b831c-bc8d-482e-8cc2-956b35abb1b4 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:38.889231 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877b831c-bc8d-482e-8cc2-956b35abb1b4 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:38.889697 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877b831c-bc8d-482e-8cc2-956b35abb1b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:38.906817 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a37507-2fac-4113-a691-710d6a63090e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:38.907748 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a37507-2fac-4113-a691-710d6a63090e None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:38.908224 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a37507-2fac-4113-a691-710d6a63090e None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:38.908665 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a37507-2fac-4113-a691-710d6a63090e None admin] PATH_INFO: `/v3/projects/3325fd0bf7e2439aa626201e2ae25c6b` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:38.910097 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a37507-2fac-4113-a691-710d6a63090e None admin] RBAC: Authorizing `identity:get_project(project_id=3325fd0bf7e2439aa626201e2ae25c6b)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:38.912275 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a37507-2fac-4113-a691-710d6a63090e None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:38.918424 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-877b831c-bc8d-482e-8cc2-956b35abb1b4 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:17:38.918892 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 126/255] 10.222.0.102 () {56 vars in 1191 bytes} [Thu Sep 1 17:17:38 2022] GET /identity/v3/projects/3325fd0bf7e2439aa626201e2ae25c6b => generated 303 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:38.920617 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-877b831c-bc8d-482e-8cc2-956b35abb1b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:17:38.923248 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a37507-2fac-4113-a691-710d6a63090e None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:38.983911 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:39.027253 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 130/256] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:17:38 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:17:39.091887 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a37507-2fac-4113-a691-710d6a63090e None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:39.140607 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb7c011-ac8b-41e5-9a40-1892fb110cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:39.141675 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb7c011-ac8b-41e5-9a40-1892fb110cba None admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:39.142156 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb7c011-ac8b-41e5-9a40-1892fb110cba None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:39.142673 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb7c011-ac8b-41e5-9a40-1892fb110cba None admin] PATH_INFO: `/v3/projects/3325fd0bf7e2439aa626201e2ae25c6b/users/0dc9f9b26c9b4337b4ecba20f6cecf5e/roles/b75cb360925e4ba5a65f14965c103ffd` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:39.144839 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb7c011-ac8b-41e5-9a40-1892fb110cba None admin] RBAC: Authorizing `identity:create_grant(project_id=3325fd0bf7e2439aa626201e2ae25c6b, user_id=0dc9f9b26c9b4337b4ecba20f6cecf5e, role_id=b75cb360925e4ba5a65f14965c103ffd)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:39.147048 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb7c011-ac8b-41e5-9a40-1892fb110cba None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:39.172398 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 127/257] 10.222.0.102 () {58 vars in 1444 bytes} [Thu Sep 1 17:17:38 2022] PUT /identity/v3/projects/3325fd0bf7e2439aa626201e2ae25c6b/users/0dc9f9b26c9b4337b4ecba20f6cecf5e/roles/b75cb360925e4ba5a65f14965c103ffd => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:17:39.320909 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877b831c-bc8d-482e-8cc2-956b35abb1b4 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:39.426082 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877b831c-bc8d-482e-8cc2-956b35abb1b4 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:39.461332 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:39.467693 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd199c74-0f50-4d4e-9fb7-7a66a802f6fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:39.468602 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd199c74-0f50-4d4e-9fb7-7a66a802f6fc None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:39.469114 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd199c74-0f50-4d4e-9fb7-7a66a802f6fc None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:39.469603 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd199c74-0f50-4d4e-9fb7-7a66a802f6fc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:39.486691 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd199c74-0f50-4d4e-9fb7-7a66a802f6fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:17:39.491968 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-bd199c74-0f50-4d4e-9fb7-7a66a802f6fc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 17:17:39.491968 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:17:39.491968 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:17:39.491968 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:17:39.491968 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:17:39.491968 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:17:39.491968 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:17:39.491968 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:17:39.491968 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:17:39.491968 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:17:39.491968 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:17:39.491968 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:17:39.491968 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:17:39.491968 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:17:39.491968 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:17:39.491968 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:17:39.492820 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:17:39.492820 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:17:39.492820 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:17:39.492820 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:17:39.492820 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:17:39.492820 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:17:39.492820 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:17:39.492820 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:17:39.492820 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:17:39.492820 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:17:39.492820 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:17:39.492820 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:17:39.492820 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:17:39.492820 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:17:39.492820 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:17:39.492820 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:17:39.493648 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:17:39.493648 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:17:39.493648 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:17:39.493648 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:17:39.493648 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:17:39.493648 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:17:39.493648 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:17:39.493648 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:17:39.493648 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:17:39.493648 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:17:39.493648 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 17:17:39.493648 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:17:39.496191 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 131/258] 10.222.0.102 () {56 vars in 1119 bytes} [Thu Sep 1 17:17:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:17:39.502068 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb7c011-ac8b-41e5-9a40-1892fb110cba None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:39.530774 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd199c74-0f50-4d4e-9fb7-7a66a802f6fc None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:39.667160 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:39.675098 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb7c011-ac8b-41e5-9a40-1892fb110cba None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:39.689238 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd199c74-0f50-4d4e-9fb7-7a66a802f6fc None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:39.736360 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05deef40-6145-4010-9c6d-cad18f9b25de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:39.737833 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05deef40-6145-4010-9c6d-cad18f9b25de None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:39.739279 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05deef40-6145-4010-9c6d-cad18f9b25de None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:39.740056 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05deef40-6145-4010-9c6d-cad18f9b25de None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:39.741403 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea27fb3a-bb38-4de7-837b-2b4f6cdfd1bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:39.742893 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea27fb3a-bb38-4de7-837b-2b4f6cdfd1bd None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:39.743538 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea27fb3a-bb38-4de7-837b-2b4f6cdfd1bd None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:39.745098 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea27fb3a-bb38-4de7-837b-2b4f6cdfd1bd None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:39.745990 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea27fb3a-bb38-4de7-837b-2b4f6cdfd1bd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:17:39.747471 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea27fb3a-bb38-4de7-837b-2b4f6cdfd1bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:39.751702 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea27fb3a-bb38-4de7-837b-2b4f6cdfd1bd None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:39.763332 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05deef40-6145-4010-9c6d-cad18f9b25de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:17:39.771324 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 128/259] 10.222.0.102 () {56 vars in 1116 bytes} [Thu Sep 1 17:17:39 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:39.772230 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-05deef40-6145-4010-9c6d-cad18f9b25de None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:17:39.772230 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:17:39.772230 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:17:39.772230 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:17:39.772230 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:17:39.772230 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:17:39.772230 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:17:39.772230 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:17:39.772230 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:17:39.772230 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:17:39.772230 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:17:39.772230 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:17:39.772230 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:17:39.772230 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:17:39.772230 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:17:39.772230 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:17:39.773064 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:17:39.773064 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:17:39.773064 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:17:39.773064 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:17:39.773064 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:17:39.773064 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:17:39.773064 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:17:39.773064 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:17:39.773064 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:17:39.773064 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:17:39.773064 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:17:39.773064 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:17:39.773064 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:17:39.773064 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:17:39.773064 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:17:39.773064 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:17:39.778217 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:17:39.778217 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:17:39.778217 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:17:39.778217 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:17:39.778217 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:17:39.778217 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:17:39.778217 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:17:39.778217 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:17:39.778217 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:17:39.778217 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:17:39.778217 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:17:39.778217 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:17:39.780290 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 132/260] 10.222.0.102 () {56 vars in 1101 bytes} [Thu Sep 1 17:17:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:17:39.783623 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea27fb3a-bb38-4de7-837b-2b4f6cdfd1bd None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:39.788051 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05deef40-6145-4010-9c6d-cad18f9b25de None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:39.870701 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:39.968573 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea27fb3a-bb38-4de7-837b-2b4f6cdfd1bd None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:39.990529 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05deef40-6145-4010-9c6d-cad18f9b25de None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:40.037824 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88e2300-c64e-42db-8255-fb998a51f323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:40.038473 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88e2300-c64e-42db-8255-fb998a51f323 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:40.038755 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88e2300-c64e-42db-8255-fb998a51f323 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:40.039032 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88e2300-c64e-42db-8255-fb998a51f323 None admin] PATH_INFO: `/v3/users/7cf4904211ee414793d522ae570a016e` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:40.046222 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88e2300-c64e-42db-8255-fb998a51f323 None admin] RBAC: Authorizing `identity:get_user(user_id=7cf4904211ee414793d522ae570a016e)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:40.047716 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88e2300-c64e-42db-8255-fb998a51f323 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:40.054048 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 129/261] 10.222.0.102 () {56 vars in 1182 bytes} [Thu Sep 1 17:17:39 2022] GET /identity/v3/users/7cf4904211ee414793d522ae570a016e => generated 284 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:40.061924 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4231ac6-a932-45a3-993f-f816152d5820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:40.062983 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4231ac6-a932-45a3-993f-f816152d5820 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:40.063459 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4231ac6-a932-45a3-993f-f816152d5820 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:40.063935 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4231ac6-a932-45a3-993f-f816152d5820 None admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:40.064841 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4231ac6-a932-45a3-993f-f816152d5820 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:17:40.065410 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4231ac6-a932-45a3-993f-f816152d5820 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:40.067558 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4231ac6-a932-45a3-993f-f816152d5820 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:40.072215 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88e2300-c64e-42db-8255-fb998a51f323 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:40.085563 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 133/262] 10.222.0.102 () {56 vars in 1104 bytes} [Thu Sep 1 17:17:39 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:40.090892 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4231ac6-a932-45a3-993f-f816152d5820 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:40.133683 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:40.218924 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4231ac6-a932-45a3-993f-f816152d5820 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:40.229636 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:40.284659 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76df1af-4534-410d-a81e-3df021b049a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:40.287189 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76df1af-4534-410d-a81e-3df021b049a2 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:40.287855 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76df1af-4534-410d-a81e-3df021b049a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:40.288888 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76df1af-4534-410d-a81e-3df021b049a2 None admin] PATH_INFO: `/v3/users/7cf4904211ee414793d522ae570a016e` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:40.297332 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76df1af-4534-410d-a81e-3df021b049a2 None admin] RBAC: Authorizing `identity:get_user(user_id=7cf4904211ee414793d522ae570a016e)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:40.299991 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76df1af-4534-410d-a81e-3df021b049a2 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:40.302678 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88e2300-c64e-42db-8255-fb998a51f323 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:40.306257 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 134/263] 10.222.0.102 () {56 vars in 1182 bytes} [Thu Sep 1 17:17:40 2022] GET /identity/v3/users/7cf4904211ee414793d522ae570a016e => generated 284 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:17:40.311799 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76df1af-4534-410d-a81e-3df021b049a2 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:40.387633 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee86781-8705-43de-bd2f-f1eeda4284b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:40.389084 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee86781-8705-43de-bd2f-f1eeda4284b6 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:40.389388 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee86781-8705-43de-bd2f-f1eeda4284b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:40.389666 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee86781-8705-43de-bd2f-f1eeda4284b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:40.526296 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee86781-8705-43de-bd2f-f1eeda4284b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:40.527975 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee86781-8705-43de-bd2f-f1eeda4284b6 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:40.562051 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76df1af-4534-410d-a81e-3df021b049a2 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:40.577662 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:40.633812 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9c692f-ead2-48a3-9adf-1032ed89b323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:40.634814 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9c692f-ead2-48a3-9adf-1032ed89b323 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:40.635294 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9c692f-ead2-48a3-9adf-1032ed89b323 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:40.635762 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9c692f-ead2-48a3-9adf-1032ed89b323 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:40.665494 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 130/264] 10.222.0.102 () {58 vars in 1287 bytes} [Thu Sep 1 17:17:40 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 594 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 17:17:40.672436 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee86781-8705-43de-bd2f-f1eeda4284b6 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:40.725701 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:40.793442 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee86781-8705-43de-bd2f-f1eeda4284b6 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:40.794103 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9c692f-ead2-48a3-9adf-1032ed89b323 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:40.795575 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9c692f-ead2-48a3-9adf-1032ed89b323 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:40.821675 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:40.829723 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331bf2f8-dbca-46aa-8781-b93fb158a68c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:40.830374 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331bf2f8-dbca-46aa-8781-b93fb158a68c None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:40.830663 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331bf2f8-dbca-46aa-8781-b93fb158a68c None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:40.830945 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331bf2f8-dbca-46aa-8781-b93fb158a68c None admin] PATH_INFO: `/v3/projects/3325fd0bf7e2439aa626201e2ae25c6b` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:40.832099 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331bf2f8-dbca-46aa-8781-b93fb158a68c None admin] RBAC: Authorizing `identity:get_project(project_id=3325fd0bf7e2439aa626201e2ae25c6b)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:40.834003 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331bf2f8-dbca-46aa-8781-b93fb158a68c None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:40.836145 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 131/265] 10.222.0.102 () {56 vars in 1191 bytes} [Thu Sep 1 17:17:40 2022] GET /identity/v3/projects/3325fd0bf7e2439aa626201e2ae25c6b => generated 303 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:40.846536 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331bf2f8-dbca-46aa-8781-b93fb158a68c None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:40.964086 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331bf2f8-dbca-46aa-8781-b93fb158a68c None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:41.011155 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c709d61-1b46-4c92-89a6-035e2bea8570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:41.011781 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c709d61-1b46-4c92-89a6-035e2bea8570 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:41.012009 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c709d61-1b46-4c92-89a6-035e2bea8570 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:41.012239 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c709d61-1b46-4c92-89a6-035e2bea8570 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:41.012966 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c709d61-1b46-4c92-89a6-035e2bea8570 None admin] RBAC: Adding query filter params (role.id=abef78c185c74c16a2c7e8a170583f9f, scope.project.id=3325fd0bf7e2439aa626201e2ae25c6b, user.id=7cf4904211ee414793d522ae570a016e) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:17:41.013271 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c709d61-1b46-4c92-89a6-035e2bea8570 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:41.014732 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c709d61-1b46-4c92-89a6-035e2bea8570 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:41.020918 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 132/266] 10.222.0.102 () {56 vars in 1379 bytes} [Thu Sep 1 17:17:40 2022] GET /identity/v3/role_assignments?user.id=7cf4904211ee414793d522ae570a016e&scope.project.id=3325fd0bf7e2439aa626201e2ae25c6b&role.id=abef78c185c74c16a2c7e8a170583f9f => generated 610 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:41.022308 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddb01ed-7593-422b-9ddc-86872dbdd5ba None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:41.022753 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddb01ed-7593-422b-9ddc-86872dbdd5ba None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:41.023162 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddb01ed-7593-422b-9ddc-86872dbdd5ba None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:41.024539 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 133/267] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:17:41 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:17:41.030292 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea73a1b-3f59-46b9-854f-dcc92585848d None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:41.030872 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea73a1b-3f59-46b9-854f-dcc92585848d None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:41.031313 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea73a1b-3f59-46b9-854f-dcc92585848d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:41.050904 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-1ea73a1b-3f59-46b9-854f-dcc92585848d None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:17:41.052304 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 135/268] 10.222.0.102 () {58 vars in 1287 bytes} [Thu Sep 1 17:17:40 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 741 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 01 17:17:41.056969 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-1ea73a1b-3f59-46b9-854f-dcc92585848d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:17:41.058293 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9c692f-ead2-48a3-9adf-1032ed89b323 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:41.114505 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:41.139575 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 134/269] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:17:41 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 17:17:41.197650 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:41.222511 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9c692f-ead2-48a3-9adf-1032ed89b323 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:41.269182 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7151da10-6c33-4998-b598-cebd7b85bda7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:41.269845 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7151da10-6c33-4998-b598-cebd7b85bda7 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:41.270653 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7151da10-6c33-4998-b598-cebd7b85bda7 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:41.270920 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7151da10-6c33-4998-b598-cebd7b85bda7 None admin] PATH_INFO: `/v3/projects/3325fd0bf7e2439aa626201e2ae25c6b` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:41.278117 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7151da10-6c33-4998-b598-cebd7b85bda7 None admin] RBAC: Authorizing `identity:get_project(project_id=3325fd0bf7e2439aa626201e2ae25c6b)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:41.279675 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7151da10-6c33-4998-b598-cebd7b85bda7 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:41.281407 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 136/270] 10.222.0.102 () {56 vars in 1191 bytes} [Thu Sep 1 17:17:41 2022] GET /identity/v3/projects/3325fd0bf7e2439aa626201e2ae25c6b => generated 303 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:41.288192 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea73a1b-3f59-46b9-854f-dcc92585848d None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:41.395706 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea73a1b-3f59-46b9-854f-dcc92585848d None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:41.430573 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7546177c-60ae-43e8-9809-515b6ff23a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:41.431194 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7546177c-60ae-43e8-9809-515b6ff23a54 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:41.431471 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7546177c-60ae-43e8-9809-515b6ff23a54 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:41.431737 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7546177c-60ae-43e8-9809-515b6ff23a54 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:41.432479 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7546177c-60ae-43e8-9809-515b6ff23a54 None admin] RBAC: Adding query filter params (role.id=36577a6f46d642d4b81805ede9872490, scope.project.id=3325fd0bf7e2439aa626201e2ae25c6b, user.id=7cf4904211ee414793d522ae570a016e) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:17:41.432778 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7546177c-60ae-43e8-9809-515b6ff23a54 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:41.434242 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7546177c-60ae-43e8-9809-515b6ff23a54 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:41.438655 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba86891-e4dd-4211-b0a4-2f7a3f9dc2e9 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:41.438950 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba86891-e4dd-4211-b0a4-2f7a3f9dc2e9 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:41.439223 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba86891-e4dd-4211-b0a4-2f7a3f9dc2e9 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:41.441124 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 137/271] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:17:41 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Sep 01 17:17:41.442685 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 135/272] 10.222.0.102 () {56 vars in 1379 bytes} [Thu Sep 1 17:17:41 2022] GET /identity/v3/role_assignments?user.id=7cf4904211ee414793d522ae570a016e&scope.project.id=3325fd0bf7e2439aa626201e2ae25c6b&role.id=36577a6f46d642d4b81805ede9872490 => generated 610 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:41.446613 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca482f5-81d1-4b68-8e51-e2d4a72ab79c None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:41.446897 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca482f5-81d1-4b68-8e51-e2d4a72ab79c None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:41.447173 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca482f5-81d1-4b68-8e51-e2d4a72ab79c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:41.467719 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-7ca482f5-81d1-4b68-8e51-e2d4a72ab79c None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:17:41.468959 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-7ca482f5-81d1-4b68-8e51-e2d4a72ab79c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:17:41.544457 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7546177c-60ae-43e8-9809-515b6ff23a54 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:41.558539 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 138/273] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:17:41 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:17:41.559871 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef433532-ae51-4f26-abaf-97af0f58a3c7 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:41.560815 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef433532-ae51-4f26-abaf-97af0f58a3c7 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:41.561089 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef433532-ae51-4f26-abaf-97af0f58a3c7 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:41.562564 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 139/274] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:17:41 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:17:41.570487 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19537ee5-f260-4509-8c09-8fdef9a3c7e1 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:41.571102 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19537ee5-f260-4509-8c09-8fdef9a3c7e1 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:41.571377 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19537ee5-f260-4509-8c09-8fdef9a3c7e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:41.603789 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-19537ee5-f260-4509-8c09-8fdef9a3c7e1 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:17:41.605783 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-19537ee5-f260-4509-8c09-8fdef9a3c7e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:17:41.645792 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:41.715473 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7546177c-60ae-43e8-9809-515b6ff23a54 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:41.717980 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 140/275] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:17:41 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:17:41.751673 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2254280c-988c-49e1-805a-e30938e55bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:41.752311 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2254280c-988c-49e1-805a-e30938e55bc4 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:41.752588 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2254280c-988c-49e1-805a-e30938e55bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:41.752857 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2254280c-988c-49e1-805a-e30938e55bc4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:41.765043 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2254280c-988c-49e1-805a-e30938e55bc4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:17:41.770488 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-2254280c-988c-49e1-805a-e30938e55bc4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 17:17:41.770488 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:17:41.770488 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:17:41.770488 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:17:41.770488 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:17:41.770488 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:17:41.770488 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:17:41.770488 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:17:41.770488 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:17:41.770488 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:17:41.770488 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:17:41.770488 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:17:41.770488 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:17:41.770488 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:17:41.770488 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:17:41.770488 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:17:41.771421 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:17:41.771421 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:17:41.771421 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:17:41.771421 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:17:41.771421 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:17:41.771421 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:17:41.771421 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:17:41.771421 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:17:41.771421 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:17:41.771421 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:17:41.771421 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:17:41.771421 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:17:41.771421 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:17:41.771421 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:17:41.771421 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:17:41.771421 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:17:41.772214 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:17:41.772214 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:17:41.772214 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:17:41.772214 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:17:41.772214 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:17:41.772214 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:17:41.772214 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:17:41.772214 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:17:41.772214 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:17:41.772214 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:17:41.772214 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 17:17:41.772214 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:17:41.776412 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 136/276] 10.222.0.102 () {56 vars in 1104 bytes} [Thu Sep 1 17:17:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:17:41.777819 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19537ee5-f260-4509-8c09-8fdef9a3c7e1 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:41.781603 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:41.896155 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2254280c-988c-49e1-805a-e30938e55bc4 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:41.900523 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19537ee5-f260-4509-8c09-8fdef9a3c7e1 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:41.943726 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf62896-e9b0-43f8-8f38-6a1e7a44d6b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:41.944342 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf62896-e9b0-43f8-8f38-6a1e7a44d6b9 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:41.944618 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf62896-e9b0-43f8-8f38-6a1e7a44d6b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:41.944887 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf62896-e9b0-43f8-8f38-6a1e7a44d6b9 None admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:41.945597 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf62896-e9b0-43f8-8f38-6a1e7a44d6b9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:17:41.945899 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf62896-e9b0-43f8-8f38-6a1e7a44d6b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:41.947261 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf62896-e9b0-43f8-8f38-6a1e7a44d6b9 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:41.957023 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 141/277] 10.222.0.102 () {56 vars in 1106 bytes} [Thu Sep 1 17:17:41 2022] GET /identity/v3/roles?name=reader => generated 343 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:41.962992 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf62896-e9b0-43f8-8f38-6a1e7a44d6b9 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:42.012833 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2254280c-988c-49e1-805a-e30938e55bc4 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:42.027934 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:42.032672 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:42.061515 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34c5081-c012-422f-990c-fea0529882c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:42.062285 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34c5081-c012-422f-990c-fea0529882c6 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:42.062562 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34c5081-c012-422f-990c-fea0529882c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:42.062937 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34c5081-c012-422f-990c-fea0529882c6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:42.076468 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34c5081-c012-422f-990c-fea0529882c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:17:42.079043 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf62896-e9b0-43f8-8f38-6a1e7a44d6b9 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:42.081387 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-f34c5081-c012-422f-990c-fea0529882c6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:17:42.081387 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:17:42.081387 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:17:42.081387 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:17:42.081387 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:17:42.081387 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:17:42.081387 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:17:42.081387 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:17:42.081387 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:17:42.081387 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:17:42.081387 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:17:42.081387 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:17:42.081387 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:17:42.081387 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:17:42.081387 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:17:42.081387 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:17:42.082215 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:17:42.082215 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:17:42.082215 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:17:42.082215 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:17:42.082215 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:17:42.082215 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:17:42.082215 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:17:42.082215 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:17:42.082215 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:17:42.082215 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:17:42.082215 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:17:42.082215 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:17:42.082215 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:17:42.082215 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:17:42.082215 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:17:42.082215 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:17:42.083017 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:17:42.083017 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:17:42.083017 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:17:42.083017 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:17:42.083017 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:17:42.083017 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:17:42.083017 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:17:42.083017 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:17:42.083017 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:17:42.083017 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:17:42.083017 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:17:42.083017 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:17:42.085451 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 137/278] 10.222.0.102 () {56 vars in 1104 bytes} [Thu Sep 1 17:17:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:17:42.086784 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34c5081-c012-422f-990c-fea0529882c6 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:42.116509 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7e7416-b764-455d-8ef4-324c5e28a67e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:42.117134 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7e7416-b764-455d-8ef4-324c5e28a67e None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:42.117458 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7e7416-b764-455d-8ef4-324c5e28a67e None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:42.117730 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7e7416-b764-455d-8ef4-324c5e28a67e None admin] PATH_INFO: `/v3/users/ef6bf9831b334481b89ffb2c3d06fdce` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:42.118870 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7e7416-b764-455d-8ef4-324c5e28a67e None admin] RBAC: Authorizing `identity:get_user(user_id=ef6bf9831b334481b89ffb2c3d06fdce)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:42.120307 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7e7416-b764-455d-8ef4-324c5e28a67e None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:42.121922 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 142/279] 10.222.0.102 () {56 vars in 1182 bytes} [Thu Sep 1 17:17:41 2022] GET /identity/v3/users/ef6bf9831b334481b89ffb2c3d06fdce => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:17:42.127098 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7e7416-b764-455d-8ef4-324c5e28a67e None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:42.198091 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34c5081-c012-422f-990c-fea0529882c6 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:42.226775 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7e7416-b764-455d-8ef4-324c5e28a67e None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:42.231822 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c519a086-86df-453a-bdab-5c80b4450994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:42.232452 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c519a086-86df-453a-bdab-5c80b4450994 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:42.232730 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c519a086-86df-453a-bdab-5c80b4450994 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:42.233006 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c519a086-86df-453a-bdab-5c80b4450994 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:42.234132 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c519a086-86df-453a-bdab-5c80b4450994 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:42.235580 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c519a086-86df-453a-bdab-5c80b4450994 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:42.237158 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 138/280] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:17:42 2022] GET /identity/v3/domains/default => generated 200 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:42.237950 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c519a086-86df-453a-bdab-5c80b4450994 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:42.258953 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:42.264200 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd59347-6457-49da-beda-587b00f0d3b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:42.264802 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd59347-6457-49da-beda-587b00f0d3b9 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:42.265080 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd59347-6457-49da-beda-587b00f0d3b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:42.265380 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd59347-6457-49da-beda-587b00f0d3b9 None admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:42.266041 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd59347-6457-49da-beda-587b00f0d3b9 None admin] RBAC: Adding query filter params (name=member) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:17:42.266342 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd59347-6457-49da-beda-587b00f0d3b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:42.267698 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd59347-6457-49da-beda-587b00f0d3b9 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:42.277242 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 143/281] 10.222.0.102 () {56 vars in 1106 bytes} [Thu Sep 1 17:17:42 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:42.278473 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af9567f-f7d4-43ca-8461-118f02e59829 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:42.278760 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af9567f-f7d4-43ca-8461-118f02e59829 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:42.279097 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af9567f-f7d4-43ca-8461-118f02e59829 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:42.280370 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 144/282] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:17:42 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:17:42.281057 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af9567f-f7d4-43ca-8461-118f02e59829 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:42.282225 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:42.340783 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c519a086-86df-453a-bdab-5c80b4450994 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:42.373984 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c85cf15-085d-4cde-83cf-3533c07dc020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:42.374615 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c85cf15-085d-4cde-83cf-3533c07dc020 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:42.374890 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c85cf15-085d-4cde-83cf-3533c07dc020 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:42.375159 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c85cf15-085d-4cde-83cf-3533c07dc020 None admin] PATH_INFO: `/v3/projects/3325fd0bf7e2439aa626201e2ae25c6b` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:42.376264 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c85cf15-085d-4cde-83cf-3533c07dc020 None admin] RBAC: Authorizing `identity:get_project(project_id=3325fd0bf7e2439aa626201e2ae25c6b)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:42.377763 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c85cf15-085d-4cde-83cf-3533c07dc020 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:42.379912 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 139/283] 10.222.0.102 () {56 vars in 1191 bytes} [Thu Sep 1 17:17:42 2022] GET /identity/v3/projects/3325fd0bf7e2439aa626201e2ae25c6b => generated 303 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:42.380674 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c85cf15-085d-4cde-83cf-3533c07dc020 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:42.388519 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af9567f-f7d4-43ca-8461-118f02e59829 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:42.422697 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66763b69-6107-4fc1-b2fb-7ab75cb082dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:42.423286 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66763b69-6107-4fc1-b2fb-7ab75cb082dc None admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:42.423569 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66763b69-6107-4fc1-b2fb-7ab75cb082dc None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:42.423840 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66763b69-6107-4fc1-b2fb-7ab75cb082dc None admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:42.424481 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66763b69-6107-4fc1-b2fb-7ab75cb082dc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:42.425960 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66763b69-6107-4fc1-b2fb-7ab75cb082dc None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:42.447830 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 145/284] 10.222.0.102 () {60 vars in 1137 bytes} [Thu Sep 1 17:17:42 2022] POST /identity/v3/users => generated 294 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:17:42.448946 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b54fd5-49cd-4b9b-b3ad-608e691b950a None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:42.449181 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b54fd5-49cd-4b9b-b3ad-608e691b950a None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:42.449485 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b54fd5-49cd-4b9b-b3ad-608e691b950a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:42.475785 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-14b54fd5-49cd-4b9b-b3ad-608e691b950a None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:17:42.477022 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-14b54fd5-49cd-4b9b-b3ad-608e691b950a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:17:42.488202 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c85cf15-085d-4cde-83cf-3533c07dc020 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:42.498426 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:42.530578 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03d8738-e5ce-4cac-8e87-d5ea84557842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:42.531230 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03d8738-e5ce-4cac-8e87-d5ea84557842 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:42.531515 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03d8738-e5ce-4cac-8e87-d5ea84557842 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:42.531805 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03d8738-e5ce-4cac-8e87-d5ea84557842 None admin] PATH_INFO: `/v3/users/08cd02fc2caa400d87badacc15189672` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:42.538358 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:42.548755 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03d8738-e5ce-4cac-8e87-d5ea84557842 None admin] RBAC: Authorizing `identity:get_user(user_id=08cd02fc2caa400d87badacc15189672)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:42.550305 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03d8738-e5ce-4cac-8e87-d5ea84557842 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:42.552250 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 140/285] 10.222.0.102 () {56 vars in 1182 bytes} [Thu Sep 1 17:17:42 2022] GET /identity/v3/users/08cd02fc2caa400d87badacc15189672 => generated 294 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:42.552869 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03d8738-e5ce-4cac-8e87-d5ea84557842 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:42.571593 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 146/286] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:17:42 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:17:42.572765 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b54fd5-49cd-4b9b-b3ad-608e691b950a None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:42.692517 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03d8738-e5ce-4cac-8e87-d5ea84557842 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:42.704791 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b54fd5-49cd-4b9b-b3ad-608e691b950a None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:42.745741 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a4cef1-6f85-4eaf-ac70-ba924b8db3d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:42.747753 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a4cef1-6f85-4eaf-ac70-ba924b8db3d9 None admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:42.748868 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a4cef1-6f85-4eaf-ac70-ba924b8db3d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:42.749911 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a4cef1-6f85-4eaf-ac70-ba924b8db3d9 None admin] PATH_INFO: `/v3/projects/3325fd0bf7e2439aa626201e2ae25c6b/users/ef6bf9831b334481b89ffb2c3d06fdce/roles/ab816d03be024513884447335f2de35a` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:42.752844 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a4cef1-6f85-4eaf-ac70-ba924b8db3d9 None admin] RBAC: Authorizing `identity:create_grant(project_id=3325fd0bf7e2439aa626201e2ae25c6b, user_id=ef6bf9831b334481b89ffb2c3d06fdce, role_id=ab816d03be024513884447335f2de35a)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:42.754968 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a4cef1-6f85-4eaf-ac70-ba924b8db3d9 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:42.759283 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60414d2-7332-44ce-8656-c03819ff4079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:42.759901 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60414d2-7332-44ce-8656-c03819ff4079 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:42.760178 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60414d2-7332-44ce-8656-c03819ff4079 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:42.760447 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60414d2-7332-44ce-8656-c03819ff4079 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:42.761292 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60414d2-7332-44ce-8656-c03819ff4079 None admin] RBAC: Adding query filter params (role.id=b75cb360925e4ba5a65f14965c103ffd, user.id=08cd02fc2caa400d87badacc15189672, scope.system=all) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:17:42.761684 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60414d2-7332-44ce-8656-c03819ff4079 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:42.763498 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60414d2-7332-44ce-8656-c03819ff4079 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:42.772682 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 141/287] 10.222.0.102 () {58 vars in 1444 bytes} [Thu Sep 1 17:17:42 2022] PUT /identity/v3/projects/3325fd0bf7e2439aa626201e2ae25c6b/users/ef6bf9831b334481b89ffb2c3d06fdce/roles/ab816d03be024513884447335f2de35a => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:17:42.773219 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 147/288] 10.222.0.102 () {56 vars in 1313 bytes} [Thu Sep 1 17:17:42 2022] GET /identity/v3/role_assignments?user.id=08cd02fc2caa400d87badacc15189672&scope.system=all&role.id=b75cb360925e4ba5a65f14965c103ffd => generated 227 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:42.849374 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a4cef1-6f85-4eaf-ac70-ba924b8db3d9 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:42.892570 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:42.985158 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a4cef1-6f85-4eaf-ac70-ba924b8db3d9 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:43.023158 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a33b4c9-7178-456a-8ade-d0c6eb769610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:43.023906 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a33b4c9-7178-456a-8ade-d0c6eb769610 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:43.024090 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a33b4c9-7178-456a-8ade-d0c6eb769610 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:43.024359 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a33b4c9-7178-456a-8ade-d0c6eb769610 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:43.035765 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a33b4c9-7178-456a-8ade-d0c6eb769610 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:17:43.040755 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-3a33b4c9-7178-456a-8ade-d0c6eb769610 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:17:43.040755 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:17:43.040755 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:17:43.040755 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:17:43.040755 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:17:43.040755 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:17:43.040755 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:17:43.040755 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:17:43.040755 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:17:43.040755 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:17:43.040755 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:17:43.040755 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:17:43.040755 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:17:43.040755 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:17:43.040755 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:17:43.040755 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:17:43.041739 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:17:43.041739 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:17:43.041739 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:17:43.041739 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:17:43.041739 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:17:43.041739 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:17:43.041739 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:17:43.041739 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:17:43.041739 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:17:43.041739 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:17:43.041739 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:17:43.041739 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:17:43.041739 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:17:43.041739 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:17:43.041739 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:17:43.041739 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:17:43.042504 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:17:43.042504 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:17:43.042504 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:17:43.042504 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:17:43.042504 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:17:43.042504 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:17:43.042504 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:17:43.042504 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:17:43.042504 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:17:43.042504 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:17:43.042504 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:17:43.042504 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:17:43.042504 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 142/289] 10.222.0.102 () {56 vars in 1104 bytes} [Thu Sep 1 17:17:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:17:43.047846 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60414d2-7332-44ce-8656-c03819ff4079 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:43.109974 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:43.175206 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60414d2-7332-44ce-8656-c03819ff4079 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:43.226025 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c911cc2-2f6f-4673-b38a-80c84811fc77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:43.226644 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c911cc2-2f6f-4673-b38a-80c84811fc77 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:43.226925 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c911cc2-2f6f-4673-b38a-80c84811fc77 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:43.227449 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c911cc2-2f6f-4673-b38a-80c84811fc77 None admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:43.228139 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c911cc2-2f6f-4673-b38a-80c84811fc77 None admin] RBAC: Adding query filter params (name=member) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:17:43.228441 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c911cc2-2f6f-4673-b38a-80c84811fc77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:43.229856 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c911cc2-2f6f-4673-b38a-80c84811fc77 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:43.240081 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 148/290] 10.222.0.102 () {56 vars in 1106 bytes} [Thu Sep 1 17:17:43 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:43.245832 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a33b4c9-7178-456a-8ade-d0c6eb769610 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:43.339219 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:43.355710 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a33b4c9-7178-456a-8ade-d0c6eb769610 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:43.396517 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc75359-833c-42f7-99f0-bbb4ec4145f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:43.397152 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc75359-833c-42f7-99f0-bbb4ec4145f7 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:43.397473 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc75359-833c-42f7-99f0-bbb4ec4145f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:43.397748 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc75359-833c-42f7-99f0-bbb4ec4145f7 None admin] PATH_INFO: `/v3/users/0dc9f9b26c9b4337b4ecba20f6cecf5e` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:43.398874 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc75359-833c-42f7-99f0-bbb4ec4145f7 None admin] RBAC: Authorizing `identity:get_user(user_id=0dc9f9b26c9b4337b4ecba20f6cecf5e)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:43.400362 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc75359-833c-42f7-99f0-bbb4ec4145f7 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:43.401989 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 143/291] 10.222.0.102 () {56 vars in 1182 bytes} [Thu Sep 1 17:17:43 2022] GET /identity/v3/users/0dc9f9b26c9b4337b4ecba20f6cecf5e => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:43.407689 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c911cc2-2f6f-4673-b38a-80c84811fc77 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:43.529786 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c911cc2-2f6f-4673-b38a-80c84811fc77 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:43.542405 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:43.583046 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5771d0f4-dab0-45b4-80b7-0227ae9b55a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:43.583663 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5771d0f4-dab0-45b4-80b7-0227ae9b55a5 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:43.583941 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5771d0f4-dab0-45b4-80b7-0227ae9b55a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:43.584214 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5771d0f4-dab0-45b4-80b7-0227ae9b55a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:43.676607 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5771d0f4-dab0-45b4-80b7-0227ae9b55a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:43.678420 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5771d0f4-dab0-45b4-80b7-0227ae9b55a5 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:43.810754 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 149/292] 10.222.0.102 () {58 vars in 1287 bytes} [Thu Sep 1 17:17:43 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 17:17:43.818494 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc75359-833c-42f7-99f0-bbb4ec4145f7 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:43.957719 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc75359-833c-42f7-99f0-bbb4ec4145f7 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:43.993829 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525416b9-5279-457a-84b4-e07be4d2828f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:43.994850 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525416b9-5279-457a-84b4-e07be4d2828f None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:43.995716 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525416b9-5279-457a-84b4-e07be4d2828f None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:43.995996 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525416b9-5279-457a-84b4-e07be4d2828f None admin] PATH_INFO: `/v3/projects/3325fd0bf7e2439aa626201e2ae25c6b` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:43.997126 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525416b9-5279-457a-84b4-e07be4d2828f None admin] RBAC: Authorizing `identity:get_project(project_id=3325fd0bf7e2439aa626201e2ae25c6b)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:43.998680 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525416b9-5279-457a-84b4-e07be4d2828f None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:44.000633 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 144/293] 10.222.0.102 () {56 vars in 1191 bytes} [Thu Sep 1 17:17:43 2022] GET /identity/v3/projects/3325fd0bf7e2439aa626201e2ae25c6b => generated 303 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:44.006597 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5771d0f4-dab0-45b4-80b7-0227ae9b55a5 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:44.015106 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:44.116608 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5771d0f4-dab0-45b4-80b7-0227ae9b55a5 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:44.154571 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c609b4c-d650-4b95-b268-c984b99030bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:44.155246 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c609b4c-d650-4b95-b268-c984b99030bb None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:44.155525 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c609b4c-d650-4b95-b268-c984b99030bb None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:44.155798 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c609b4c-d650-4b95-b268-c984b99030bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:44.156580 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c609b4c-d650-4b95-b268-c984b99030bb None admin] RBAC: Adding query filter params (role.id=b75cb360925e4ba5a65f14965c103ffd, scope.project.id=3325fd0bf7e2439aa626201e2ae25c6b, user.id=0dc9f9b26c9b4337b4ecba20f6cecf5e) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:17:44.156893 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c609b4c-d650-4b95-b268-c984b99030bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:44.158448 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c609b4c-d650-4b95-b268-c984b99030bb None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:44.165729 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 150/294] 10.222.0.102 () {56 vars in 1379 bytes} [Thu Sep 1 17:17:44 2022] GET /identity/v3/role_assignments?user.id=0dc9f9b26c9b4337b4ecba20f6cecf5e&scope.project.id=3325fd0bf7e2439aa626201e2ae25c6b&role.id=b75cb360925e4ba5a65f14965c103ffd => generated 610 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:44.938381 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3497f284-9481-449a-a22b-9e0b7470fcb8 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:44.938887 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3497f284-9481-449a-a22b-9e0b7470fcb8 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:44.938959 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3497f284-9481-449a-a22b-9e0b7470fcb8 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:44.940506 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 145/295] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:17:44 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:17:44.946471 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbb7fa5-4b0f-4b63-a99f-5e1888863f3b None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:44.946757 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbb7fa5-4b0f-4b63-a99f-5e1888863f3b None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:44.947036 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbb7fa5-4b0f-4b63-a99f-5e1888863f3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:44.965609 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-1cbb7fa5-4b0f-4b63-a99f-5e1888863f3b None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:17:44.966896 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-1cbb7fa5-4b0f-4b63-a99f-5e1888863f3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:17:45.031936 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:45.037120 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 151/296] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:17:44 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 17:17:45.116197 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56065fd3-ed40-435f-b0c8-b8d402185782 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:45.116489 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56065fd3-ed40-435f-b0c8-b8d402185782 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:45.116736 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56065fd3-ed40-435f-b0c8-b8d402185782 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:45.118397 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 146/297] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:17:45 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:17:45.123659 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bb34cd-379b-4f9f-9d00-47cc2c3e5614 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:45.124471 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bb34cd-379b-4f9f-9d00-47cc2c3e5614 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:45.124913 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bb34cd-379b-4f9f-9d00-47cc2c3e5614 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:45.131598 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d39cd36-8a21-417e-ab4d-daa6e2a44c5a None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:45.131952 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d39cd36-8a21-417e-ab4d-daa6e2a44c5a None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:45.132300 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d39cd36-8a21-417e-ab4d-daa6e2a44c5a None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:45.133886 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 147/298] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:17:45 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:17:45.138966 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f72d5e-fe49-4718-b9a0-ea2f1dc2b796 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:45.139322 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f72d5e-fe49-4718-b9a0-ea2f1dc2b796 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:45.139671 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f72d5e-fe49-4718-b9a0-ea2f1dc2b796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:45.145255 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-64bb34cd-379b-4f9f-9d00-47cc2c3e5614 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:17:45.146642 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-64bb34cd-379b-4f9f-9d00-47cc2c3e5614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:17:45.158667 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-19f72d5e-fe49-4718-b9a0-ea2f1dc2b796 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:17:45.159946 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-19f72d5e-fe49-4718-b9a0-ea2f1dc2b796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:17:45.180462 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:45.233364 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 152/299] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:17:45 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:17:45.234032 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f8aca5-b02a-452b-aa0a-02ee2a4266f2 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:45.234317 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f8aca5-b02a-452b-aa0a-02ee2a4266f2 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:45.234619 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f8aca5-b02a-452b-aa0a-02ee2a4266f2 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:45.240523 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 153/300] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:17:45 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:17:45.241764 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37de70e-0354-4e5f-8bd4-ec8342436909 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:45.242043 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37de70e-0354-4e5f-8bd4-ec8342436909 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:45.242312 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37de70e-0354-4e5f-8bd4-ec8342436909 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:45.243338 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 148/301] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:17:45 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:17:45.261039 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-b37de70e-0354-4e5f-8bd4-ec8342436909 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:17:45.262266 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-b37de70e-0354-4e5f-8bd4-ec8342436909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:17:45.327158 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f72d5e-fe49-4718-b9a0-ea2f1dc2b796 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:45.335130 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 154/302] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:17:45 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:17:45.442145 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f72d5e-fe49-4718-b9a0-ea2f1dc2b796 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:45.476039 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbc7aa2-ba32-4c6c-9651-52c4e6d773d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:45.476666 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbc7aa2-ba32-4c6c-9651-52c4e6d773d9 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:45.476943 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbc7aa2-ba32-4c6c-9651-52c4e6d773d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:45.477210 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbc7aa2-ba32-4c6c-9651-52c4e6d773d9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:45.478621 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbc7aa2-ba32-4c6c-9651-52c4e6d773d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:45.480062 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbc7aa2-ba32-4c6c-9651-52c4e6d773d9 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:45.481964 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 149/303] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:17:45 2022] GET /identity/v3/domains/default => generated 200 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:17:45.487564 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37de70e-0354-4e5f-8bd4-ec8342436909 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:45.516078 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbc7aa2-ba32-4c6c-9651-52c4e6d773d9 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:45.540151 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:45.553991 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:45.596260 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37de70e-0354-4e5f-8bd4-ec8342436909 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:45.632150 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ee502d-93fa-413d-9145-e78267d749f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:45.632838 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ee502d-93fa-413d-9145-e78267d749f9 None admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:45.633121 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ee502d-93fa-413d-9145-e78267d749f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:45.633423 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ee502d-93fa-413d-9145-e78267d749f9 None admin] PATH_INFO: `/v3/groups` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:45.634060 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ee502d-93fa-413d-9145-e78267d749f9 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:45.634587 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbc7aa2-ba32-4c6c-9651-52c4e6d773d9 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:45.635495 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ee502d-93fa-413d-9145-e78267d749f9 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:45.652352 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 155/304] 10.222.0.102 () {60 vars in 1139 bytes} [Thu Sep 1 17:17:45 2022] POST /identity/v3/groups => generated 230 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:17:45.653750 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ee502d-93fa-413d-9145-e78267d749f9 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:45.678112 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505236c6-dfe2-4206-b396-3bc776138e23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:45.679055 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505236c6-dfe2-4206-b396-3bc776138e23 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:45.679335 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505236c6-dfe2-4206-b396-3bc776138e23 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:45.679598 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505236c6-dfe2-4206-b396-3bc776138e23 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:45.691310 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505236c6-dfe2-4206-b396-3bc776138e23 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:17:45.696652 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-505236c6-dfe2-4206-b396-3bc776138e23 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 17:17:45.696652 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:17:45.696652 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:17:45.696652 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:17:45.696652 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:17:45.696652 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:17:45.696652 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:17:45.696652 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:17:45.696652 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:17:45.696652 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:17:45.696652 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:17:45.696652 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:17:45.696652 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:17:45.696652 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:17:45.696652 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:17:45.696652 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:17:45.697669 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:17:45.697669 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:17:45.697669 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:17:45.697669 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:17:45.697669 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:17:45.697669 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:17:45.697669 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:17:45.697669 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:17:45.697669 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:17:45.697669 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:17:45.697669 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:17:45.697669 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:17:45.697669 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:17:45.697669 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:17:45.697669 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:17:45.697669 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:17:45.698466 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:17:45.698466 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:17:45.698466 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:17:45.698466 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:17:45.698466 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:17:45.698466 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:17:45.698466 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:17:45.698466 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:17:45.698466 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:17:45.698466 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:17:45.698466 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 17:17:45.698466 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:17:45.698466 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 150/305] 10.222.0.102 () {56 vars in 1104 bytes} [Thu Sep 1 17:17:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:17:45.699216 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505236c6-dfe2-4206-b396-3bc776138e23 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:45.785051 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ee502d-93fa-413d-9145-e78267d749f9 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:45.795676 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:45.809385 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:45.820866 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505236c6-dfe2-4206-b396-3bc776138e23 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:45.832929 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70955285-b22b-4d90-8e58-117088cf1c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:45.833607 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70955285-b22b-4d90-8e58-117088cf1c70 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:45.833885 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70955285-b22b-4d90-8e58-117088cf1c70 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:45.834153 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70955285-b22b-4d90-8e58-117088cf1c70 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:45.845407 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70955285-b22b-4d90-8e58-117088cf1c70 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:17:45.852167 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-70955285-b22b-4d90-8e58-117088cf1c70 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 17:17:45.852167 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:17:45.852167 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:17:45.852167 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:17:45.852167 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:17:45.852167 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:17:45.852167 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:17:45.852167 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:17:45.852167 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:17:45.852167 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:17:45.852167 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:17:45.852167 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:17:45.852167 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:17:45.852167 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:17:45.852167 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:17:45.852167 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:17:45.853729 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:17:45.853729 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:17:45.853729 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:17:45.853729 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:17:45.853729 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:17:45.853729 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:17:45.853729 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:17:45.853729 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:17:45.853729 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:17:45.853729 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:17:45.853729 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:17:45.853729 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:17:45.853729 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:17:45.853729 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:17:45.853729 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:17:45.853729 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:17:45.854524 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:17:45.854524 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:17:45.854524 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:17:45.854524 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:17:45.854524 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:17:45.854524 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:17:45.854524 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:17:45.854524 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:17:45.854524 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:17:45.854524 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:17:45.854524 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 17:17:45.854524 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:17:45.854524 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 156/306] 10.222.0.102 () {56 vars in 1104 bytes} [Thu Sep 1 17:17:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:17:45.855184 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70955285-b22b-4d90-8e58-117088cf1c70 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:45.864501 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb92688-ce0e-40ba-a3c3-afbf7b67f7a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:45.865138 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb92688-ce0e-40ba-a3c3-afbf7b67f7a9 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:45.865443 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb92688-ce0e-40ba-a3c3-afbf7b67f7a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:45.865711 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb92688-ce0e-40ba-a3c3-afbf7b67f7a9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:45.876532 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb92688-ce0e-40ba-a3c3-afbf7b67f7a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:17:45.881410 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-1fb92688-ce0e-40ba-a3c3-afbf7b67f7a9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:17:45.881410 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:17:45.881410 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:17:45.881410 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:17:45.881410 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:17:45.881410 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:17:45.881410 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:17:45.881410 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:17:45.881410 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:17:45.881410 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:17:45.881410 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:17:45.881410 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:17:45.881410 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:17:45.881410 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:17:45.881410 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:17:45.881410 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:17:45.882238 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:17:45.882238 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:17:45.882238 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:17:45.882238 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:17:45.882238 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:17:45.882238 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:17:45.882238 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:17:45.882238 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:17:45.882238 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:17:45.882238 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:17:45.882238 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:17:45.882238 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:17:45.882238 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:17:45.882238 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:17:45.882238 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:17:45.882238 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:17:45.883043 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:17:45.883043 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:17:45.883043 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:17:45.883043 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:17:45.883043 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:17:45.883043 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:17:45.883043 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:17:45.883043 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:17:45.883043 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:17:45.883043 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:17:45.883043 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:17:45.883043 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:17:45.883043 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 151/307] 10.222.0.102 () {56 vars in 1104 bytes} [Thu Sep 1 17:17:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:17:45.883691 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb92688-ce0e-40ba-a3c3-afbf7b67f7a9 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:45.967747 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70955285-b22b-4d90-8e58-117088cf1c70 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:45.995376 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb92688-ce0e-40ba-a3c3-afbf7b67f7a9 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:46.007263 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537453b5-1245-4fd2-ac23-aec6372fcb42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:46.008173 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537453b5-1245-4fd2-ac23-aec6372fcb42 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:46.008643 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537453b5-1245-4fd2-ac23-aec6372fcb42 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:46.009147 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537453b5-1245-4fd2-ac23-aec6372fcb42 None admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:46.010012 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537453b5-1245-4fd2-ac23-aec6372fcb42 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:17:46.010519 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537453b5-1245-4fd2-ac23-aec6372fcb42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:46.012159 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537453b5-1245-4fd2-ac23-aec6372fcb42 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:46.029054 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 157/308] 10.222.0.102 () {56 vars in 1106 bytes} [Thu Sep 1 17:17:45 2022] GET /identity/v3/roles?name=reader => generated 343 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:46.030028 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537453b5-1245-4fd2-ac23-aec6372fcb42 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:46.035204 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dcc071-e76f-44d1-b44a-3389f03ba497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:46.035826 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dcc071-e76f-44d1-b44a-3389f03ba497 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:46.036106 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dcc071-e76f-44d1-b44a-3389f03ba497 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:46.036370 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dcc071-e76f-44d1-b44a-3389f03ba497 None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:46.037029 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dcc071-e76f-44d1-b44a-3389f03ba497 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:17:46.037351 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dcc071-e76f-44d1-b44a-3389f03ba497 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:46.038707 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dcc071-e76f-44d1-b44a-3389f03ba497 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:46.047796 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 152/309] 10.222.0.102 () {56 vars in 1106 bytes} [Thu Sep 1 17:17:45 2022] GET /identity/v3/roles?name=reader => generated 343 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:46.053393 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dcc071-e76f-44d1-b44a-3389f03ba497 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:46.056174 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:46.078360 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:46.144497 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537453b5-1245-4fd2-ac23-aec6372fcb42 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:46.161235 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dcc071-e76f-44d1-b44a-3389f03ba497 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:46.183995 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0750638-24b2-4cdf-bb81-0af8d5a07669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:46.184943 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0750638-24b2-4cdf-bb81-0af8d5a07669 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:46.185478 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0750638-24b2-4cdf-bb81-0af8d5a07669 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:46.185594 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0750638-24b2-4cdf-bb81-0af8d5a07669 None admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:46.186239 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0750638-24b2-4cdf-bb81-0af8d5a07669 None admin] RBAC: Adding query filter params (name=member) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:17:46.186547 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0750638-24b2-4cdf-bb81-0af8d5a07669 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:46.188078 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0750638-24b2-4cdf-bb81-0af8d5a07669 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:46.198000 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5c640a-e156-4132-a96a-c43400efa974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:46.198632 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5c640a-e156-4132-a96a-c43400efa974 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:46.198908 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5c640a-e156-4132-a96a-c43400efa974 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:46.199177 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5c640a-e156-4132-a96a-c43400efa974 None admin] PATH_INFO: `/v3/users/ef6bf9831b334481b89ffb2c3d06fdce` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:46.200299 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5c640a-e156-4132-a96a-c43400efa974 None admin] RBAC: Authorizing `identity:get_user(user_id=ef6bf9831b334481b89ffb2c3d06fdce)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:46.201771 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5c640a-e156-4132-a96a-c43400efa974 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:46.202554 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 158/310] 10.222.0.102 () {56 vars in 1106 bytes} [Thu Sep 1 17:17:46 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:46.203206 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0750638-24b2-4cdf-bb81-0af8d5a07669 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:46.207913 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 153/311] 10.222.0.102 () {56 vars in 1182 bytes} [Thu Sep 1 17:17:46 2022] GET /identity/v3/users/ef6bf9831b334481b89ffb2c3d06fdce => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:46.208577 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5c640a-e156-4132-a96a-c43400efa974 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:46.287396 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:46.314642 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0750638-24b2-4cdf-bb81-0af8d5a07669 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:46.317611 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:46.321662 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5c640a-e156-4132-a96a-c43400efa974 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:46.353161 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e17bb5-0220-412a-9bff-20573ec77b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:46.353815 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e17bb5-0220-412a-9bff-20573ec77b34 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:46.354094 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e17bb5-0220-412a-9bff-20573ec77b34 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:46.354474 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e17bb5-0220-412a-9bff-20573ec77b34 None admin] PATH_INFO: `/v3/users/6b2665ec5f804ecfa24c8d986daea4ad` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:46.357479 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c54e65b-7de0-4751-b7ae-e562728ffca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:46.358115 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c54e65b-7de0-4751-b7ae-e562728ffca6 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:46.358397 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c54e65b-7de0-4751-b7ae-e562728ffca6 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:46.358660 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c54e65b-7de0-4751-b7ae-e562728ffca6 None admin] PATH_INFO: `/v3/users/08cd02fc2caa400d87badacc15189672` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:46.359774 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c54e65b-7de0-4751-b7ae-e562728ffca6 None admin] RBAC: Authorizing `identity:get_user(user_id=08cd02fc2caa400d87badacc15189672)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:46.361226 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c54e65b-7de0-4751-b7ae-e562728ffca6 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:46.362825 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 154/312] 10.222.0.102 () {56 vars in 1182 bytes} [Thu Sep 1 17:17:46 2022] GET /identity/v3/users/08cd02fc2caa400d87badacc15189672 => generated 294 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:46.363666 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c54e65b-7de0-4751-b7ae-e562728ffca6 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:46.374184 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e17bb5-0220-412a-9bff-20573ec77b34 None admin] RBAC: Authorizing `identity:get_user(user_id=6b2665ec5f804ecfa24c8d986daea4ad)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:46.375914 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e17bb5-0220-412a-9bff-20573ec77b34 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:46.378444 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 159/313] 10.222.0.102 () {56 vars in 1182 bytes} [Thu Sep 1 17:17:46 2022] GET /identity/v3/users/6b2665ec5f804ecfa24c8d986daea4ad => generated 294 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:46.380014 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e17bb5-0220-412a-9bff-20573ec77b34 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:46.480649 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c54e65b-7de0-4751-b7ae-e562728ffca6 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:46.484069 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e17bb5-0220-412a-9bff-20573ec77b34 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:46.516352 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c67733-a501-4a14-84e4-ac8632477ff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:46.516966 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c67733-a501-4a14-84e4-ac8632477ff7 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:46.517242 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c67733-a501-4a14-84e4-ac8632477ff7 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:46.517541 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c67733-a501-4a14-84e4-ac8632477ff7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:46.522248 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1838cb-24b6-4094-8981-e8363f959bfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:46.523097 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1838cb-24b6-4094-8981-e8363f959bfa None admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:46.524098 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1838cb-24b6-4094-8981-e8363f959bfa None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:46.524556 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1838cb-24b6-4094-8981-e8363f959bfa None admin] PATH_INFO: `/v3/system/users/08cd02fc2caa400d87badacc15189672/roles/b75cb360925e4ba5a65f14965c103ffd` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:46.526285 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1838cb-24b6-4094-8981-e8363f959bfa None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=08cd02fc2caa400d87badacc15189672, role_id=b75cb360925e4ba5a65f14965c103ffd)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:46.526982 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:46.527866 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1838cb-24b6-4094-8981-e8363f959bfa None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:46.539482 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 160/314] 10.222.0.102 () {58 vars in 1339 bytes} [Thu Sep 1 17:17:46 2022] PUT /identity/v3/system/users/08cd02fc2caa400d87badacc15189672/roles/b75cb360925e4ba5a65f14965c103ffd => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:17:46.540671 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1838cb-24b6-4094-8981-e8363f959bfa None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:46.589207 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:46.592987 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c67733-a501-4a14-84e4-ac8632477ff7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:46.594924 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c67733-a501-4a14-84e4-ac8632477ff7 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:46.699010 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1838cb-24b6-4094-8981-e8363f959bfa None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:46.717484 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 155/315] 10.222.0.102 () {58 vars in 1287 bytes} [Thu Sep 1 17:17:46 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 17:17:46.725379 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c67733-a501-4a14-84e4-ac8632477ff7 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:46.742307 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becae481-b263-44d7-ae10-233cf122eab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:46.743015 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becae481-b263-44d7-ae10-233cf122eab0 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:46.743361 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becae481-b263-44d7-ae10-233cf122eab0 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:46.743800 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becae481-b263-44d7-ae10-233cf122eab0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:46.744612 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becae481-b263-44d7-ae10-233cf122eab0 None admin] RBAC: Adding query filter params (role.id=ab816d03be024513884447335f2de35a, user.id=6b2665ec5f804ecfa24c8d986daea4ad, scope.system=all) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:17:46.745137 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becae481-b263-44d7-ae10-233cf122eab0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:46.746691 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becae481-b263-44d7-ae10-233cf122eab0 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:46.752491 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 161/316] 10.222.0.102 () {56 vars in 1313 bytes} [Thu Sep 1 17:17:46 2022] GET /identity/v3/role_assignments?user.id=6b2665ec5f804ecfa24c8d986daea4ad&scope.system=all&role.id=ab816d03be024513884447335f2de35a => generated 227 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:46.774126 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:46.846767 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c67733-a501-4a14-84e4-ac8632477ff7 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:46.887590 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1211a3fa-9931-46ea-a477-2a7dd04e9086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:46.888552 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1211a3fa-9931-46ea-a477-2a7dd04e9086 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:46.889047 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1211a3fa-9931-46ea-a477-2a7dd04e9086 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:46.889515 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1211a3fa-9931-46ea-a477-2a7dd04e9086 None admin] PATH_INFO: `/v3/projects/3325fd0bf7e2439aa626201e2ae25c6b` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:46.891070 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1211a3fa-9931-46ea-a477-2a7dd04e9086 None admin] RBAC: Authorizing `identity:get_project(project_id=3325fd0bf7e2439aa626201e2ae25c6b)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:46.892723 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1211a3fa-9931-46ea-a477-2a7dd04e9086 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:46.895058 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 156/317] 10.222.0.102 () {56 vars in 1191 bytes} [Thu Sep 1 17:17:46 2022] GET /identity/v3/projects/3325fd0bf7e2439aa626201e2ae25c6b => generated 303 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:46.903587 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becae481-b263-44d7-ae10-233cf122eab0 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:47.001866 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:47.012410 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becae481-b263-44d7-ae10-233cf122eab0 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:47.048400 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32c052f-e85e-40a1-b22d-c9c08fbee86c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:47.049002 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32c052f-e85e-40a1-b22d-c9c08fbee86c None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:47.049276 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32c052f-e85e-40a1-b22d-c9c08fbee86c None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:47.049571 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32c052f-e85e-40a1-b22d-c9c08fbee86c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:47.050305 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32c052f-e85e-40a1-b22d-c9c08fbee86c None admin] RBAC: Adding query filter params (role.id=ab816d03be024513884447335f2de35a, scope.project.id=3325fd0bf7e2439aa626201e2ae25c6b, user.id=ef6bf9831b334481b89ffb2c3d06fdce) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:17:47.050601 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32c052f-e85e-40a1-b22d-c9c08fbee86c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:47.052628 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32c052f-e85e-40a1-b22d-c9c08fbee86c None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:47.059084 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 162/318] 10.222.0.102 () {56 vars in 1379 bytes} [Thu Sep 1 17:17:46 2022] GET /identity/v3/role_assignments?user.id=ef6bf9831b334481b89ffb2c3d06fdce&scope.project.id=3325fd0bf7e2439aa626201e2ae25c6b&role.id=ab816d03be024513884447335f2de35a => generated 610 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:47.615634 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f204aed8-8311-442b-8ee4-050b0c69a478 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:47.616175 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f204aed8-8311-442b-8ee4-050b0c69a478 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:47.616175 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f204aed8-8311-442b-8ee4-050b0c69a478 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:47.617817 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 157/319] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:17:47 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:17:47.622919 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43e0aca-e6c3-4c0d-9f3f-54cf18375b62 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:47.623337 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43e0aca-e6c3-4c0d-9f3f-54cf18375b62 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:47.623501 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43e0aca-e6c3-4c0d-9f3f-54cf18375b62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:47.642145 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-e43e0aca-e6c3-4c0d-9f3f-54cf18375b62 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:17:47.643346 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-e43e0aca-e6c3-4c0d-9f3f-54cf18375b62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:17:47.715745 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 163/320] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:17:47 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:17:47.983214 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f204aed8-8311-442b-8ee4-050b0c69a478 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:48.084622 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f204aed8-8311-442b-8ee4-050b0c69a478 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:48.092482 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:48.120467 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b8f580-5490-4ad8-bb3e-4c115557ffd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:48.121105 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b8f580-5490-4ad8-bb3e-4c115557ffd8 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:48.121411 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b8f580-5490-4ad8-bb3e-4c115557ffd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:48.121688 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b8f580-5490-4ad8-bb3e-4c115557ffd8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:48.122784 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b8f580-5490-4ad8-bb3e-4c115557ffd8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:48.124238 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b8f580-5490-4ad8-bb3e-4c115557ffd8 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:48.126002 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 158/321] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:17:47 2022] GET /identity/v3/domains/default => generated 200 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:48.131404 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43e0aca-e6c3-4c0d-9f3f-54cf18375b62 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:48.217776 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:48.235607 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43e0aca-e6c3-4c0d-9f3f-54cf18375b62 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:48.270574 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bea2c81-6322-4267-be48-b9fd8a977e51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:48.271182 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bea2c81-6322-4267-be48-b9fd8a977e51 None admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:48.271461 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bea2c81-6322-4267-be48-b9fd8a977e51 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:48.271738 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bea2c81-6322-4267-be48-b9fd8a977e51 None admin] PATH_INFO: `/v3/groups` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:48.272372 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bea2c81-6322-4267-be48-b9fd8a977e51 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:48.273837 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bea2c81-6322-4267-be48-b9fd8a977e51 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:48.290581 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 164/322] 10.222.0.102 () {60 vars in 1139 bytes} [Thu Sep 1 17:17:48 2022] POST /identity/v3/groups => generated 227 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:17:48.566694 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d7559e-19ae-4a69-baab-be7a516a00ea None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:48.567090 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d7559e-19ae-4a69-baab-be7a516a00ea None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:48.567284 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d7559e-19ae-4a69-baab-be7a516a00ea None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:48.568948 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 159/323] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:17:48 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:17:48.574353 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1c754d-99ed-4c07-b66d-478ca6ffbbb3 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:48.574640 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1c754d-99ed-4c07-b66d-478ca6ffbbb3 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:48.574920 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1c754d-99ed-4c07-b66d-478ca6ffbbb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:48.593416 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-1a1c754d-99ed-4c07-b66d-478ca6ffbbb3 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:17:48.595418 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-1a1c754d-99ed-4c07-b66d-478ca6ffbbb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:17:48.669684 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 165/324] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:17:48 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:17:48.708739 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62f3817-0b71-4b46-8e9a-032ea1a85ca8 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:48.709236 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62f3817-0b71-4b46-8e9a-032ea1a85ca8 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:48.709399 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62f3817-0b71-4b46-8e9a-032ea1a85ca8 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:48.710927 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 160/325] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:17:48 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:17:48.716137 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea973d8-758d-4c3b-ba8a-c3a6314aad79 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:48.716529 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea973d8-758d-4c3b-ba8a-c3a6314aad79 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:48.716813 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea973d8-758d-4c3b-ba8a-c3a6314aad79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:48.750396 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-fea973d8-758d-4c3b-ba8a-c3a6314aad79 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:17:48.751602 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-fea973d8-758d-4c3b-ba8a-c3a6314aad79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:17:48.828133 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 166/326] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:17:48 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:17:48.980060 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62f3817-0b71-4b46-8e9a-032ea1a85ca8 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:49.104079 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62f3817-0b71-4b46-8e9a-032ea1a85ca8 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:49.118056 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea973d8-758d-4c3b-ba8a-c3a6314aad79 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:49.128633 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:49.152476 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f4a8b6-5044-452b-9482-986c10faefe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:49.153117 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f4a8b6-5044-452b-9482-986c10faefe0 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:49.153417 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f4a8b6-5044-452b-9482-986c10faefe0 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:49.153691 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f4a8b6-5044-452b-9482-986c10faefe0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:49.168829 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f4a8b6-5044-452b-9482-986c10faefe0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:17:49.175076 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-f0f4a8b6-5044-452b-9482-986c10faefe0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:17:49.175076 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:17:49.175076 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:17:49.175076 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:17:49.175076 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:17:49.175076 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:17:49.175076 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:17:49.175076 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:17:49.175076 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:17:49.175076 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:17:49.175076 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:17:49.175076 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:17:49.175076 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:17:49.175076 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:17:49.175076 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:17:49.175076 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:17:49.175967 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:17:49.175967 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:17:49.175967 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:17:49.175967 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:17:49.175967 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:17:49.175967 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:17:49.175967 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:17:49.175967 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:17:49.175967 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:17:49.175967 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:17:49.175967 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:17:49.175967 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:17:49.175967 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:17:49.175967 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:17:49.175967 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:17:49.175967 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:17:49.177835 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:17:49.177835 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:17:49.177835 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:17:49.177835 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:17:49.177835 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:17:49.177835 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:17:49.177835 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:17:49.177835 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:17:49.177835 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:17:49.177835 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:17:49.177835 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:17:49.177835 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:17:49.179332 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 161/327] 10.222.0.102 () {56 vars in 1104 bytes} [Thu Sep 1 17:17:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:17:49.185300 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f4a8b6-5044-452b-9482-986c10faefe0 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:49.229852 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:49.267497 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea973d8-758d-4c3b-ba8a-c3a6314aad79 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:49.313656 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f4a8b6-5044-452b-9482-986c10faefe0 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:49.353338 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226b6649-4251-408c-9269-ecfede4c5e09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:49.353942 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226b6649-4251-408c-9269-ecfede4c5e09 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:49.354270 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226b6649-4251-408c-9269-ecfede4c5e09 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:49.354580 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226b6649-4251-408c-9269-ecfede4c5e09 None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:49.354925 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c796472-388a-44f0-961f-637e78537431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:49.355238 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226b6649-4251-408c-9269-ecfede4c5e09 None admin] RBAC: Adding query filter params (name=member) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:17:49.355623 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c796472-388a-44f0-961f-637e78537431 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:49.355787 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226b6649-4251-408c-9269-ecfede4c5e09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:49.355933 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c796472-388a-44f0-961f-637e78537431 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:49.356155 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c796472-388a-44f0-961f-637e78537431 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:49.357273 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226b6649-4251-408c-9269-ecfede4c5e09 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:49.370999 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 162/328] 10.222.0.102 () {56 vars in 1106 bytes} [Thu Sep 1 17:17:49 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:17:49.377985 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226b6649-4251-408c-9269-ecfede4c5e09 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:49.385198 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c796472-388a-44f0-961f-637e78537431 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:17:49.399691 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-2c796472-388a-44f0-961f-637e78537431 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 17:17:49.399691 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:17:49.399691 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:17:49.399691 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:17:49.399691 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:17:49.399691 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:17:49.399691 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:17:49.399691 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:17:49.399691 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:17:49.399691 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:17:49.399691 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:17:49.399691 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:17:49.399691 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:17:49.399691 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:17:49.399691 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:17:49.399691 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:17:49.401880 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:17:49.401880 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:17:49.401880 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:17:49.401880 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:17:49.401880 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:17:49.401880 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:17:49.401880 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:17:49.401880 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:17:49.401880 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:17:49.401880 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:17:49.401880 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:17:49.401880 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:17:49.401880 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:17:49.401880 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:17:49.401880 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:17:49.401880 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:17:49.403818 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:17:49.403818 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:17:49.403818 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:17:49.403818 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:17:49.403818 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:17:49.403818 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:17:49.403818 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:17:49.403818 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:17:49.403818 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:17:49.403818 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:17:49.403818 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 17:17:49.403818 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:17:49.407859 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 167/329] 10.222.0.102 () {56 vars in 1104 bytes} [Thu Sep 1 17:17:49 2022] GET /identity/v3/roles/reader => generated 84 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:17:49.431472 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c796472-388a-44f0-961f-637e78537431 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:49.561248 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:49.575840 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:49.589357 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226b6649-4251-408c-9269-ecfede4c5e09 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:49.638971 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c796472-388a-44f0-961f-637e78537431 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:49.658792 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe2bcf1-cec6-465e-9df8-b0d7e27b383e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:49.659433 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe2bcf1-cec6-465e-9df8-b0d7e27b383e None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:49.659718 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe2bcf1-cec6-465e-9df8-b0d7e27b383e None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:49.659990 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe2bcf1-cec6-465e-9df8-b0d7e27b383e None admin] PATH_INFO: `/v3/users/08cd02fc2caa400d87badacc15189672` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:49.661958 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe2bcf1-cec6-465e-9df8-b0d7e27b383e None admin] RBAC: Authorizing `identity:get_user(user_id=08cd02fc2caa400d87badacc15189672)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:49.663398 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe2bcf1-cec6-465e-9df8-b0d7e27b383e None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:49.665288 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 163/330] 10.222.0.102 () {56 vars in 1182 bytes} [Thu Sep 1 17:17:49 2022] GET /identity/v3/users/08cd02fc2caa400d87badacc15189672 => generated 294 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:49.671176 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe2bcf1-cec6-465e-9df8-b0d7e27b383e None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:49.683583 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defe918a-e38a-45d4-ab5f-29f1992d1e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:49.684236 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defe918a-e38a-45d4-ab5f-29f1992d1e8c None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:49.684452 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defe918a-e38a-45d4-ab5f-29f1992d1e8c None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:49.684722 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defe918a-e38a-45d4-ab5f-29f1992d1e8c None admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:49.687548 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defe918a-e38a-45d4-ab5f-29f1992d1e8c None admin] RBAC: Adding query filter params (name=reader) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:17:49.687879 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defe918a-e38a-45d4-ab5f-29f1992d1e8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:49.689427 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defe918a-e38a-45d4-ab5f-29f1992d1e8c None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:49.699385 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 168/331] 10.222.0.102 () {56 vars in 1106 bytes} [Thu Sep 1 17:17:49 2022] GET /identity/v3/roles?name=reader => generated 343 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:17:49.705113 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defe918a-e38a-45d4-ab5f-29f1992d1e8c None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:49.800191 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe2bcf1-cec6-465e-9df8-b0d7e27b383e None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:49.828787 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defe918a-e38a-45d4-ab5f-29f1992d1e8c None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:49.840896 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0c3095-1617-4e7f-8931-4d37fb9bf6a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:49.841551 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0c3095-1617-4e7f-8931-4d37fb9bf6a0 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:49.841831 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0c3095-1617-4e7f-8931-4d37fb9bf6a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:49.842104 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0c3095-1617-4e7f-8931-4d37fb9bf6a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:49.842838 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0c3095-1617-4e7f-8931-4d37fb9bf6a0 None admin] RBAC: Adding query filter params (role.id=b75cb360925e4ba5a65f14965c103ffd, user.id=08cd02fc2caa400d87badacc15189672, scope.system=all) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:17:49.843143 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0c3095-1617-4e7f-8931-4d37fb9bf6a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:49.844591 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0c3095-1617-4e7f-8931-4d37fb9bf6a0 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:49.858511 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 164/332] 10.222.0.102 () {56 vars in 1313 bytes} [Thu Sep 1 17:17:49 2022] GET /identity/v3/role_assignments?user.id=08cd02fc2caa400d87badacc15189672&scope.system=all&role.id=b75cb360925e4ba5a65f14965c103ffd => generated 512 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:17:49.858775 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:49.875602 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d448b8e4-920c-4f68-bade-fc6dc6a48f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:49.876299 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d448b8e4-920c-4f68-bade-fc6dc6a48f97 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:49.876577 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d448b8e4-920c-4f68-bade-fc6dc6a48f97 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:49.876851 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d448b8e4-920c-4f68-bade-fc6dc6a48f97 None admin] PATH_INFO: `/v3/users/6b2665ec5f804ecfa24c8d986daea4ad` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:49.878152 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d448b8e4-920c-4f68-bade-fc6dc6a48f97 None admin] RBAC: Authorizing `identity:get_user(user_id=6b2665ec5f804ecfa24c8d986daea4ad)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:49.879677 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d448b8e4-920c-4f68-bade-fc6dc6a48f97 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:49.881572 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 169/333] 10.222.0.102 () {56 vars in 1182 bytes} [Thu Sep 1 17:17:49 2022] GET /identity/v3/users/6b2665ec5f804ecfa24c8d986daea4ad => generated 294 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:49.887279 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0c3095-1617-4e7f-8931-4d37fb9bf6a0 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:49.921780 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:50.006955 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0c3095-1617-4e7f-8931-4d37fb9bf6a0 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:50.045759 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5076e4c4-6e98-4fcd-abfc-3a17302fe40b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:50.046414 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5076e4c4-6e98-4fcd-abfc-3a17302fe40b None admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:50.046691 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5076e4c4-6e98-4fcd-abfc-3a17302fe40b None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:50.046964 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5076e4c4-6e98-4fcd-abfc-3a17302fe40b None admin] PATH_INFO: `/v3/system/users/6b2665ec5f804ecfa24c8d986daea4ad/roles/ab816d03be024513884447335f2de35a` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:50.048394 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5076e4c4-6e98-4fcd-abfc-3a17302fe40b None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=6b2665ec5f804ecfa24c8d986daea4ad, role_id=ab816d03be024513884447335f2de35a)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:50.049806 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5076e4c4-6e98-4fcd-abfc-3a17302fe40b None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:50.060391 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 165/334] 10.222.0.102 () {58 vars in 1339 bytes} [Thu Sep 1 17:17:49 2022] PUT /identity/v3/system/users/6b2665ec5f804ecfa24c8d986daea4ad/roles/ab816d03be024513884447335f2de35a => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:17:50.634183 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbbc17e-e762-4934-88f7-40cf3900f5cf None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:50.634563 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbbc17e-e762-4934-88f7-40cf3900f5cf None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:50.634840 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbbc17e-e762-4934-88f7-40cf3900f5cf None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:50.636340 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 170/335] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:17:50 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:17:50.641991 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74704fc6-6e4f-47e0-9e65-c1b8000dc460 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:50.642572 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74704fc6-6e4f-47e0-9e65-c1b8000dc460 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:50.643047 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74704fc6-6e4f-47e0-9e65-c1b8000dc460 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:50.663823 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-74704fc6-6e4f-47e0-9e65-c1b8000dc460 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:17:50.665451 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-74704fc6-6e4f-47e0-9e65-c1b8000dc460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:17:50.750164 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 166/336] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:17:50 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:17:51.122663 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05593fce-c557-4347-b6bb-9bb5303d952b None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:51.123673 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05593fce-c557-4347-b6bb-9bb5303d952b None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:51.124179 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05593fce-c557-4347-b6bb-9bb5303d952b None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:51.125979 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 171/337] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:17:51 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:17:51.131711 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933b0910-baaf-439a-9d43-b534e9d8cd13 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:51.132566 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933b0910-baaf-439a-9d43-b534e9d8cd13 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:51.133041 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933b0910-baaf-439a-9d43-b534e9d8cd13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:51.159403 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-933b0910-baaf-439a-9d43-b534e9d8cd13 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:17:51.161531 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-933b0910-baaf-439a-9d43-b534e9d8cd13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:17:51.180194 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:51.253235 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 167/338] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:17:51 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 17:17:51.362688 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05593fce-c557-4347-b6bb-9bb5303d952b None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:51.502948 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05593fce-c557-4347-b6bb-9bb5303d952b None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:51.554964 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1182eb7a-36d5-4377-8a6b-3c64958f3aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:51.555966 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1182eb7a-36d5-4377-8a6b-3c64958f3aef None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:51.556471 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1182eb7a-36d5-4377-8a6b-3c64958f3aef None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:51.556965 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1182eb7a-36d5-4377-8a6b-3c64958f3aef None admin] PATH_INFO: `/v3/roles/member` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:51.570612 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1182eb7a-36d5-4377-8a6b-3c64958f3aef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:17:51.575840 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-1182eb7a-36d5-4377-8a6b-3c64958f3aef None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:17:51.575840 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:17:51.575840 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:17:51.575840 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:17:51.575840 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:17:51.575840 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:17:51.575840 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:17:51.575840 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:17:51.575840 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:17:51.575840 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:17:51.575840 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:17:51.575840 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:17:51.575840 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:17:51.575840 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:17:51.575840 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:17:51.575840 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:17:51.576717 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:17:51.576717 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:17:51.576717 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:17:51.576717 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:17:51.576717 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:17:51.576717 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:17:51.576717 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:17:51.576717 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:17:51.576717 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:17:51.576717 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:17:51.576717 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:17:51.576717 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:17:51.576717 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:17:51.576717 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:17:51.576717 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:17:51.576717 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:17:51.577565 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:17:51.577565 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:17:51.577565 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:17:51.577565 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:17:51.577565 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:17:51.577565 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:17:51.577565 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:17:51.577565 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:17:51.577565 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:17:51.577565 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:17:51.577565 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:17:51.577565 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:17:51.580145 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 172/339] 10.222.0.102 () {56 vars in 1104 bytes} [Thu Sep 1 17:17:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:17:51.586183 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933b0910-baaf-439a-9d43-b534e9d8cd13 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:51.611406 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf876628-92e8-4cbe-8944-3638c20d8767 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:51.611707 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf876628-92e8-4cbe-8944-3638c20d8767 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:51.611953 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf876628-92e8-4cbe-8944-3638c20d8767 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:51.613288 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 173/340] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:17:51 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Sep 01 17:17:51.621780 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b62688e-cc85-4b09-888c-09187d3d70fd None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:51.622060 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b62688e-cc85-4b09-888c-09187d3d70fd None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:51.622882 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b62688e-cc85-4b09-888c-09187d3d70fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:51.644471 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-5b62688e-cc85-4b09-888c-09187d3d70fd None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:17:51.645776 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-5b62688e-cc85-4b09-888c-09187d3d70fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:17:51.708144 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:51.731616 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 174/341] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:17:51 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 17:17:51.784990 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933b0910-baaf-439a-9d43-b534e9d8cd13 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:51.825999 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff662ca1-37fc-4a44-a0a0-c3c06f50db91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:51.826624 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff662ca1-37fc-4a44-a0a0-c3c06f50db91 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:51.826903 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff662ca1-37fc-4a44-a0a0-c3c06f50db91 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:51.827167 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff662ca1-37fc-4a44-a0a0-c3c06f50db91 None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:51.827831 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff662ca1-37fc-4a44-a0a0-c3c06f50db91 None admin] RBAC: Adding query filter params (name=member) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:17:51.828130 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff662ca1-37fc-4a44-a0a0-c3c06f50db91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:51.829527 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff662ca1-37fc-4a44-a0a0-c3c06f50db91 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:51.839171 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 168/342] 10.222.0.102 () {56 vars in 1106 bytes} [Thu Sep 1 17:17:51 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:51.844763 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b62688e-cc85-4b09-888c-09187d3d70fd None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:51.880117 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff662ca1-37fc-4a44-a0a0-c3c06f50db91 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:51.900159 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:51.977876 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b62688e-cc85-4b09-888c-09187d3d70fd None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:52.017692 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8477d5b1-e99e-4fc7-8ecf-43e24ba757a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:52.018948 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8477d5b1-e99e-4fc7-8ecf-43e24ba757a4 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:52.019406 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8477d5b1-e99e-4fc7-8ecf-43e24ba757a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:52.019814 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8477d5b1-e99e-4fc7-8ecf-43e24ba757a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:52.086406 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:52.105320 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff662ca1-37fc-4a44-a0a0-c3c06f50db91 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:52.106654 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8477d5b1-e99e-4fc7-8ecf-43e24ba757a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:52.108491 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8477d5b1-e99e-4fc7-8ecf-43e24ba757a4 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:52.156464 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774c14a6-65e8-4bf3-935c-53420ac94d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:52.157742 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774c14a6-65e8-4bf3-935c-53420ac94d40 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:52.158165 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774c14a6-65e8-4bf3-935c-53420ac94d40 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:52.158578 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774c14a6-65e8-4bf3-935c-53420ac94d40 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:52.174684 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774c14a6-65e8-4bf3-935c-53420ac94d40 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:17:52.180390 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-774c14a6-65e8-4bf3-935c-53420ac94d40 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 17:17:52.180390 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:17:52.180390 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:17:52.180390 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:17:52.180390 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:17:52.180390 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:17:52.180390 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:17:52.180390 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:17:52.180390 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:17:52.180390 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:17:52.180390 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:17:52.180390 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:17:52.180390 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:17:52.180390 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:17:52.180390 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:17:52.180390 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:17:52.185588 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:17:52.185588 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:17:52.185588 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:17:52.185588 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:17:52.185588 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:17:52.185588 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:17:52.185588 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:17:52.185588 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:17:52.185588 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:17:52.185588 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:17:52.185588 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:17:52.185588 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:17:52.185588 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:17:52.185588 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:17:52.185588 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:17:52.185588 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:17:52.186583 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:17:52.186583 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:17:52.186583 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:17:52.186583 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:17:52.186583 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:17:52.186583 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:17:52.186583 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:17:52.186583 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:17:52.186583 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:17:52.186583 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:17:52.186583 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 17:17:52.186583 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:17:52.188744 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 169/343] 10.222.0.102 () {56 vars in 1119 bytes} [Thu Sep 1 17:17:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:17:52.189749 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774c14a6-65e8-4bf3-935c-53420ac94d40 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:52.223233 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 175/344] 10.222.0.102 () {58 vars in 1287 bytes} [Thu Sep 1 17:17:51 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 17:17:52.224095 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8477d5b1-e99e-4fc7-8ecf-43e24ba757a4 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:52.282452 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:52.320368 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:52.338761 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774c14a6-65e8-4bf3-935c-53420ac94d40 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:52.341184 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8477d5b1-e99e-4fc7-8ecf-43e24ba757a4 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:52.380180 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8094067-635c-4a13-b3e6-bb0c32e10ae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:52.381072 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8094067-635c-4a13-b3e6-bb0c32e10ae3 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:52.381550 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8094067-635c-4a13-b3e6-bb0c32e10ae3 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:52.381957 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8094067-635c-4a13-b3e6-bb0c32e10ae3 None admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:52.382776 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8094067-635c-4a13-b3e6-bb0c32e10ae3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:17:52.383312 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8094067-635c-4a13-b3e6-bb0c32e10ae3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:52.385164 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8094067-635c-4a13-b3e6-bb0c32e10ae3 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:52.391879 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb2da37-8e5d-4c3c-8e21-61803145556a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:52.392337 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb2da37-8e5d-4c3c-8e21-61803145556a None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:52.392626 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb2da37-8e5d-4c3c-8e21-61803145556a None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:52.392904 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb2da37-8e5d-4c3c-8e21-61803145556a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:52.396740 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 176/345] 10.222.0.102 () {56 vars in 1116 bytes} [Thu Sep 1 17:17:52 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:52.397949 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8094067-635c-4a13-b3e6-bb0c32e10ae3 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:52.412131 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb2da37-8e5d-4c3c-8e21-61803145556a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:17:52.425656 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-bcb2da37-8e5d-4c3c-8e21-61803145556a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:17:52.425656 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:17:52.425656 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:17:52.425656 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:17:52.425656 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:17:52.425656 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:17:52.425656 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:17:52.425656 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:17:52.425656 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:17:52.425656 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:17:52.425656 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:17:52.425656 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:17:52.425656 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:17:52.425656 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:17:52.425656 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:17:52.425656 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:17:52.427427 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:17:52.427427 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:17:52.427427 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:17:52.427427 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:17:52.427427 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:17:52.427427 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:17:52.427427 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:17:52.427427 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:17:52.427427 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:17:52.427427 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:17:52.427427 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:17:52.427427 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:17:52.427427 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:17:52.427427 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:17:52.427427 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:17:52.427427 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:17:52.428349 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:17:52.428349 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:17:52.428349 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:17:52.428349 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:17:52.428349 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:17:52.428349 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:17:52.428349 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:17:52.428349 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:17:52.428349 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:17:52.428349 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:17:52.428349 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:17:52.428349 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:17:52.428349 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 170/346] 10.222.0.102 () {56 vars in 1104 bytes} [Thu Sep 1 17:17:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:17:52.428349 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb2da37-8e5d-4c3c-8e21-61803145556a None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:52.519299 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8094067-635c-4a13-b3e6-bb0c32e10ae3 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:52.558420 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef74c32a-c735-4c68-ba30-6241b5f6f99c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:52.559320 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef74c32a-c735-4c68-ba30-6241b5f6f99c None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:52.559749 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef74c32a-c735-4c68-ba30-6241b5f6f99c None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:52.560157 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef74c32a-c735-4c68-ba30-6241b5f6f99c None admin] PATH_INFO: `/v3/projects/8e88c67b942f45108c95a833547f9bc8` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:52.561702 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef74c32a-c735-4c68-ba30-6241b5f6f99c None admin] RBAC: Authorizing `identity:get_project(project_id=8e88c67b942f45108c95a833547f9bc8)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:52.563277 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef74c32a-c735-4c68-ba30-6241b5f6f99c None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:52.565292 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 177/347] 10.222.0.102 () {56 vars in 1191 bytes} [Thu Sep 1 17:17:52 2022] GET /identity/v3/projects/8e88c67b942f45108c95a833547f9bc8 => generated 299 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:52.566282 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef74c32a-c735-4c68-ba30-6241b5f6f99c None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:52.582103 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb2da37-8e5d-4c3c-8e21-61803145556a None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:52.584591 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:52.622994 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:52.634783 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff95520-3009-455a-ac58-ae0db06c50d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:52.635787 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff95520-3009-455a-ac58-ae0db06c50d7 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:52.636175 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff95520-3009-455a-ac58-ae0db06c50d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:52.636913 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff95520-3009-455a-ac58-ae0db06c50d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:52.680482 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef74c32a-c735-4c68-ba30-6241b5f6f99c None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:52.724968 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44836d31-ebc3-480f-806c-80d533fd2e37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:52.726100 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44836d31-ebc3-480f-806c-80d533fd2e37 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:52.726383 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44836d31-ebc3-480f-806c-80d533fd2e37 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:52.726656 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44836d31-ebc3-480f-806c-80d533fd2e37 None admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:52.727846 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44836d31-ebc3-480f-806c-80d533fd2e37 None admin] RBAC: Adding query filter params (name=member) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:17:52.728156 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44836d31-ebc3-480f-806c-80d533fd2e37 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:52.728489 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff95520-3009-455a-ac58-ae0db06c50d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:52.729563 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44836d31-ebc3-480f-806c-80d533fd2e37 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:52.730249 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff95520-3009-455a-ac58-ae0db06c50d7 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:52.740041 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 178/348] 10.222.0.102 () {56 vars in 1106 bytes} [Thu Sep 1 17:17:52 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:52.741267 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de2c7c1-b69a-4460-8b99-d3f2748bfb72 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:52.742004 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de2c7c1-b69a-4460-8b99-d3f2748bfb72 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:52.742388 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de2c7c1-b69a-4460-8b99-d3f2748bfb72 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:52.748771 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 179/349] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:17:52 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:17:52.749676 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de2c7c1-b69a-4460-8b99-d3f2748bfb72 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:52.860928 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:52.875927 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 171/350] 10.222.0.102 () {58 vars in 1287 bytes} [Thu Sep 1 17:17:52 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 17:17:52.875927 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff95520-3009-455a-ac58-ae0db06c50d7 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:52.897932 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de2c7c1-b69a-4460-8b99-d3f2748bfb72 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:52.907983 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:52.937748 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ade544-1a7a-45dc-8a85-d8092f1c8c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:52.938649 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ade544-1a7a-45dc-8a85-d8092f1c8c8f None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:52.939069 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ade544-1a7a-45dc-8a85-d8092f1c8c8f None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:52.939571 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ade544-1a7a-45dc-8a85-d8092f1c8c8f None admin] PATH_INFO: `/v3/groups/300c8dafaf184bf59b61b6f89bdcf7de` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:52.947061 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ade544-1a7a-45dc-8a85-d8092f1c8c8f None admin] RBAC: Authorizing `identity:get_group(group_id=300c8dafaf184bf59b61b6f89bdcf7de)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:52.949442 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ade544-1a7a-45dc-8a85-d8092f1c8c8f None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:52.951801 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 180/351] 10.222.0.102 () {56 vars in 1185 bytes} [Thu Sep 1 17:17:52 2022] GET /identity/v3/groups/300c8dafaf184bf59b61b6f89bdcf7de => generated 227 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:52.953183 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4378322d-ab3e-4398-b929-690f88f3cf98 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:52.953677 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4378322d-ab3e-4398-b929-690f88f3cf98 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:52.954106 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4378322d-ab3e-4398-b929-690f88f3cf98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:52.974041 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-4378322d-ab3e-4398-b929-690f88f3cf98 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:17:52.975527 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-4378322d-ab3e-4398-b929-690f88f3cf98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:17:52.993773 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff95520-3009-455a-ac58-ae0db06c50d7 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:53.030677 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ca3070-07dc-4224-a669-5df7c0412fd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:53.031504 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ca3070-07dc-4224-a669-5df7c0412fd9 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:53.031878 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ca3070-07dc-4224-a669-5df7c0412fd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:53.032245 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ca3070-07dc-4224-a669-5df7c0412fd9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:53.054233 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 181/352] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:17:52 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:17:53.055570 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4378322d-ab3e-4398-b929-690f88f3cf98 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:53.105623 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ca3070-07dc-4224-a669-5df7c0412fd9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:53.107096 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ca3070-07dc-4224-a669-5df7c0412fd9 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:53.139071 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:53.143219 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:53.180500 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4378322d-ab3e-4398-b929-690f88f3cf98 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:53.218466 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60354f77-a72c-47e7-80b7-82de5532d9e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:53.219270 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60354f77-a72c-47e7-80b7-82de5532d9e1 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:53.219646 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60354f77-a72c-47e7-80b7-82de5532d9e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:53.220019 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60354f77-a72c-47e7-80b7-82de5532d9e1 None admin] PATH_INFO: `/v3/projects/8e88c67b942f45108c95a833547f9bc8` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:53.221264 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60354f77-a72c-47e7-80b7-82de5532d9e1 None admin] RBAC: Authorizing `identity:get_project(project_id=8e88c67b942f45108c95a833547f9bc8)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:53.221905 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 172/353] 10.222.0.102 () {58 vars in 1287 bytes} [Thu Sep 1 17:17:52 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 17:17:53.222794 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ca3070-07dc-4224-a669-5df7c0412fd9 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:53.223149 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60354f77-a72c-47e7-80b7-82de5532d9e1 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:53.225211 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 182/354] 10.222.0.102 () {56 vars in 1191 bytes} [Thu Sep 1 17:17:53 2022] GET /identity/v3/projects/8e88c67b942f45108c95a833547f9bc8 => generated 299 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:53.232407 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60354f77-a72c-47e7-80b7-82de5532d9e1 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:53.350770 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ca3070-07dc-4224-a669-5df7c0412fd9 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:53.364667 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60354f77-a72c-47e7-80b7-82de5532d9e1 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:53.381867 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:53.393218 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7dd622-5e34-4b0a-8705-bba76318fa6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:53.393801 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7dd622-5e34-4b0a-8705-bba76318fa6c None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:53.394120 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7dd622-5e34-4b0a-8705-bba76318fa6c None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:53.394338 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7dd622-5e34-4b0a-8705-bba76318fa6c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:53.395717 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7dd622-5e34-4b0a-8705-bba76318fa6c None admin] RBAC: Adding query filter params (group.id=300c8dafaf184bf59b61b6f89bdcf7de, role.id=b75cb360925e4ba5a65f14965c103ffd, scope.project.id=8e88c67b942f45108c95a833547f9bc8) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:17:53.396026 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7dd622-5e34-4b0a-8705-bba76318fa6c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:53.397593 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7dd622-5e34-4b0a-8705-bba76318fa6c None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:53.405386 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 173/355] 10.222.0.102 () {56 vars in 1381 bytes} [Thu Sep 1 17:17:53 2022] GET /identity/v3/role_assignments?group.id=300c8dafaf184bf59b61b6f89bdcf7de&scope.project.id=8e88c67b942f45108c95a833547f9bc8&role.id=b75cb360925e4ba5a65f14965c103ffd => generated 261 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:53.406221 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7dd622-5e34-4b0a-8705-bba76318fa6c None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:53.409877 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c8db37-9fd9-4f5e-8494-b8cc35233ef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:53.411321 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c8db37-9fd9-4f5e-8494-b8cc35233ef1 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:53.411747 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c8db37-9fd9-4f5e-8494-b8cc35233ef1 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:53.412158 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c8db37-9fd9-4f5e-8494-b8cc35233ef1 None admin] PATH_INFO: `/v3/groups/300c8dafaf184bf59b61b6f89bdcf7de` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:53.413653 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c8db37-9fd9-4f5e-8494-b8cc35233ef1 None admin] RBAC: Authorizing `identity:get_group(group_id=300c8dafaf184bf59b61b6f89bdcf7de)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:53.415715 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c8db37-9fd9-4f5e-8494-b8cc35233ef1 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:53.420517 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 183/356] 10.222.0.102 () {56 vars in 1185 bytes} [Thu Sep 1 17:17:53 2022] GET /identity/v3/groups/300c8dafaf184bf59b61b6f89bdcf7de => generated 227 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:53.425905 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c8db37-9fd9-4f5e-8494-b8cc35233ef1 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:53.441731 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:53.545873 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c8db37-9fd9-4f5e-8494-b8cc35233ef1 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:53.581986 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7dd622-5e34-4b0a-8705-bba76318fa6c None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:53.587048 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc4eace-52f8-48e6-9423-c0114b141791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:53.587871 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc4eace-52f8-48e6-9423-c0114b141791 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:53.588295 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc4eace-52f8-48e6-9423-c0114b141791 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:53.588707 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc4eace-52f8-48e6-9423-c0114b141791 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:53.589703 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc4eace-52f8-48e6-9423-c0114b141791 None admin] RBAC: Adding query filter params (group.id=300c8dafaf184bf59b61b6f89bdcf7de, role.id=abef78c185c74c16a2c7e8a170583f9f, scope.project.id=8e88c67b942f45108c95a833547f9bc8) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:17:53.590184 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc4eace-52f8-48e6-9423-c0114b141791 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:53.591789 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc4eace-52f8-48e6-9423-c0114b141791 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:53.600071 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 184/357] 10.222.0.102 () {56 vars in 1381 bytes} [Thu Sep 1 17:17:53 2022] GET /identity/v3/role_assignments?group.id=300c8dafaf184bf59b61b6f89bdcf7de&scope.project.id=8e88c67b942f45108c95a833547f9bc8&role.id=abef78c185c74c16a2c7e8a170583f9f => generated 261 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:17:53.601710 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1be416-a1b0-4af1-a6b9-a4b312a6cd25 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:53.602140 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1be416-a1b0-4af1-a6b9-a4b312a6cd25 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:53.602881 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1be416-a1b0-4af1-a6b9-a4b312a6cd25 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:53.604890 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 185/358] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:17:53 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:17:53.606112 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1be416-a1b0-4af1-a6b9-a4b312a6cd25 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:53.674019 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861623e1-5ccd-4bab-be3d-7cf080b45ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:53.674847 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861623e1-5ccd-4bab-be3d-7cf080b45ab1 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:53.675182 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861623e1-5ccd-4bab-be3d-7cf080b45ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:53.675527 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861623e1-5ccd-4bab-be3d-7cf080b45ab1 None admin] PATH_INFO: `/v3/projects/3325fd0bf7e2439aa626201e2ae25c6b` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:53.677903 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861623e1-5ccd-4bab-be3d-7cf080b45ab1 None admin] RBAC: Authorizing `identity:get_project(project_id=3325fd0bf7e2439aa626201e2ae25c6b)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:53.679436 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861623e1-5ccd-4bab-be3d-7cf080b45ab1 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:53.682655 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 174/359] 10.222.0.102 () {56 vars in 1191 bytes} [Thu Sep 1 17:17:53 2022] GET /identity/v3/projects/3325fd0bf7e2439aa626201e2ae25c6b => generated 303 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:53.684559 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a530c340-6cc8-45dd-86cb-6d1ee50f8140 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:53.685048 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a530c340-6cc8-45dd-86cb-6d1ee50f8140 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:53.685575 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a530c340-6cc8-45dd-86cb-6d1ee50f8140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:53.704078 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:53.711790 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-a530c340-6cc8-45dd-86cb-6d1ee50f8140 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:17:53.713057 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-a530c340-6cc8-45dd-86cb-6d1ee50f8140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:17:53.753751 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1be416-a1b0-4af1-a6b9-a4b312a6cd25 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:53.799843 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 175/360] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:17:53 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:17:53.800415 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd725cc-ad7c-43d5-bc30-bbe77f1fbda6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:53.800902 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291895ad-628e-4c73-821b-eb4c18882de7 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:53.801244 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291895ad-628e-4c73-821b-eb4c18882de7 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:53.802237 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd725cc-ad7c-43d5-bc30-bbe77f1fbda6 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:53.802698 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd725cc-ad7c-43d5-bc30-bbe77f1fbda6 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:53.803155 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd725cc-ad7c-43d5-bc30-bbe77f1fbda6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:53.805446 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291895ad-628e-4c73-821b-eb4c18882de7 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:53.812229 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 176/361] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:17:53 2022] GET /identity => generated 271 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:17:53.813081 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291895ad-628e-4c73-821b-eb4c18882de7 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:53.815866 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd725cc-ad7c-43d5-bc30-bbe77f1fbda6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:17:53.817641 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:53.821708 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-6dd725cc-ad7c-43d5-bc30-bbe77f1fbda6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 17:17:53.821708 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:17:53.821708 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:17:53.821708 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:17:53.821708 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:17:53.821708 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:17:53.821708 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:17:53.821708 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:17:53.821708 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:17:53.821708 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:17:53.821708 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:17:53.821708 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:17:53.821708 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:17:53.821708 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:17:53.821708 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:17:53.821708 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:17:53.822618 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:17:53.822618 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:17:53.822618 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:17:53.822618 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:17:53.822618 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:17:53.822618 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:17:53.822618 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:17:53.822618 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:17:53.822618 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:17:53.822618 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:17:53.822618 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:17:53.822618 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:17:53.822618 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:17:53.822618 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:17:53.822618 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:17:53.822618 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:17:53.823440 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:17:53.823440 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:17:53.823440 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:17:53.823440 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:17:53.823440 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:17:53.823440 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:17:53.823440 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:17:53.823440 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:17:53.823440 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:17:53.823440 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:17:53.823440 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 17:17:53.823440 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:17:53.828504 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 186/362] 10.222.0.102 () {56 vars in 1119 bytes} [Thu Sep 1 17:17:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:17:53.830099 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a4a8ff-b21e-4eb4-96a5-ea7a9928d491 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:53.831099 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a4a8ff-b21e-4eb4-96a5-ea7a9928d491 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:53.831597 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a4a8ff-b21e-4eb4-96a5-ea7a9928d491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:53.853227 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-e2a4a8ff-b21e-4eb4-96a5-ea7a9928d491 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:17:53.854819 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-e2a4a8ff-b21e-4eb4-96a5-ea7a9928d491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:17:53.974242 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 187/363] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:17:53 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:17:53.978607 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a4a8ff-b21e-4eb4-96a5-ea7a9928d491 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:54.028100 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291895ad-628e-4c73-821b-eb4c18882de7 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:54.068724 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edadf278-c516-49be-869b-3022c96ece8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:54.070088 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edadf278-c516-49be-869b-3022c96ece8a None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:54.070665 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edadf278-c516-49be-869b-3022c96ece8a None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:54.071140 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edadf278-c516-49be-869b-3022c96ece8a None admin] PATH_INFO: `/v3/groups/300c8dafaf184bf59b61b6f89bdcf7de` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:54.072505 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edadf278-c516-49be-869b-3022c96ece8a None admin] RBAC: Authorizing `identity:get_group(group_id=300c8dafaf184bf59b61b6f89bdcf7de)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:54.074247 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edadf278-c516-49be-869b-3022c96ece8a None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:54.076096 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 177/364] 10.222.0.102 () {56 vars in 1185 bytes} [Thu Sep 1 17:17:53 2022] GET /identity/v3/groups/300c8dafaf184bf59b61b6f89bdcf7de => generated 227 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:54.081812 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edadf278-c516-49be-869b-3022c96ece8a None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:54.105693 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:54.162846 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:54.169589 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a4a8ff-b21e-4eb4-96a5-ea7a9928d491 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:54.210048 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edadf278-c516-49be-869b-3022c96ece8a None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:54.247396 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6a8e3b-4608-4ed7-a9c6-d77ba6ce6679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:54.248651 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6a8e3b-4608-4ed7-a9c6-d77ba6ce6679 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:54.249074 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6a8e3b-4608-4ed7-a9c6-d77ba6ce6679 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:54.249508 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6a8e3b-4608-4ed7-a9c6-d77ba6ce6679 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:54.250597 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6a8e3b-4608-4ed7-a9c6-d77ba6ce6679 None admin] RBAC: Adding query filter params (group.id=300c8dafaf184bf59b61b6f89bdcf7de, role.id=b75cb360925e4ba5a65f14965c103ffd, scope.project.id=3325fd0bf7e2439aa626201e2ae25c6b) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:17:54.251129 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6a8e3b-4608-4ed7-a9c6-d77ba6ce6679 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:54.252692 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6a8e3b-4608-4ed7-a9c6-d77ba6ce6679 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:54.254993 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc331b5-e014-49f8-bcf4-7e4ba7c3a04e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:54.255601 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc331b5-e014-49f8-bcf4-7e4ba7c3a04e None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:54.255884 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc331b5-e014-49f8-bcf4-7e4ba7c3a04e None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:54.256151 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc331b5-e014-49f8-bcf4-7e4ba7c3a04e None admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:54.256813 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc331b5-e014-49f8-bcf4-7e4ba7c3a04e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:17:54.257114 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc331b5-e014-49f8-bcf4-7e4ba7c3a04e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:54.258588 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc331b5-e014-49f8-bcf4-7e4ba7c3a04e None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:54.260923 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 178/365] 10.222.0.102 () {56 vars in 1381 bytes} [Thu Sep 1 17:17:54 2022] GET /identity/v3/role_assignments?group.id=300c8dafaf184bf59b61b6f89bdcf7de&scope.project.id=3325fd0bf7e2439aa626201e2ae25c6b&role.id=b75cb360925e4ba5a65f14965c103ffd => generated 261 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:54.262003 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6a8e3b-4608-4ed7-a9c6-d77ba6ce6679 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:54.273162 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 188/366] 10.222.0.102 () {56 vars in 1116 bytes} [Thu Sep 1 17:17:53 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:54.278733 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc331b5-e014-49f8-bcf4-7e4ba7c3a04e None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:54.440257 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6a8e3b-4608-4ed7-a9c6-d77ba6ce6679 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:54.466961 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:54.482361 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4b0881-06a1-431f-95c0-071b561ee9fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:54.483228 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4b0881-06a1-431f-95c0-071b561ee9fd None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:54.483651 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4b0881-06a1-431f-95c0-071b561ee9fd None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:54.484061 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4b0881-06a1-431f-95c0-071b561ee9fd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:54.488028 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc331b5-e014-49f8-bcf4-7e4ba7c3a04e None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:54.497774 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4b0881-06a1-431f-95c0-071b561ee9fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:17:54.504322 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-6f4b0881-06a1-431f-95c0-071b561ee9fd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:17:54.504322 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:17:54.504322 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:17:54.504322 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:17:54.504322 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:17:54.504322 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:17:54.504322 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:17:54.504322 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:17:54.504322 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:17:54.504322 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:17:54.504322 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:17:54.504322 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:17:54.504322 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:17:54.504322 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:17:54.504322 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:17:54.504322 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:17:54.505210 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:17:54.505210 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:17:54.505210 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:17:54.505210 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:17:54.505210 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:17:54.505210 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:17:54.505210 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:17:54.505210 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:17:54.505210 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:17:54.505210 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:17:54.505210 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:17:54.505210 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:17:54.505210 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:17:54.505210 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:17:54.505210 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:17:54.505210 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:17:54.506055 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:17:54.506055 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:17:54.506055 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:17:54.506055 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:17:54.506055 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:17:54.506055 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:17:54.506055 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:17:54.506055 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:17:54.506055 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:17:54.506055 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:17:54.506055 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:17:54.506055 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:17:54.508366 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 179/367] 10.222.0.102 () {56 vars in 1101 bytes} [Thu Sep 1 17:17:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:17:54.509552 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4b0881-06a1-431f-95c0-071b561ee9fd None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:54.517216 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:54.533804 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b969b3a-6143-4fe4-9530-f9b04441779e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:54.534415 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b969b3a-6143-4fe4-9530-f9b04441779e None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:54.534699 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b969b3a-6143-4fe4-9530-f9b04441779e None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:54.534960 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b969b3a-6143-4fe4-9530-f9b04441779e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:54.612503 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b969b3a-6143-4fe4-9530-f9b04441779e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:54.615067 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b969b3a-6143-4fe4-9530-f9b04441779e None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:54.725644 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 189/368] 10.222.0.102 () {58 vars in 1287 bytes} [Thu Sep 1 17:17:54 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 01 17:17:54.726548 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b969b3a-6143-4fe4-9530-f9b04441779e None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:54.757994 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:54.773100 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4b0881-06a1-431f-95c0-071b561ee9fd None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:54.834616 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b969b3a-6143-4fe4-9530-f9b04441779e None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:54.846063 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdfe5dc-dbea-443c-9722-2fddd0e83a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:54.846735 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdfe5dc-dbea-443c-9722-2fddd0e83a78 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:54.847013 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdfe5dc-dbea-443c-9722-2fddd0e83a78 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:54.847285 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdfe5dc-dbea-443c-9722-2fddd0e83a78 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:54.868542 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdfe5dc-dbea-443c-9722-2fddd0e83a78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:17:54.869573 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5930ca17-a5c0-48e9-af3e-89991db5287a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:54.870186 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5930ca17-a5c0-48e9-af3e-89991db5287a None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:54.870461 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5930ca17-a5c0-48e9-af3e-89991db5287a None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:54.870728 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5930ca17-a5c0-48e9-af3e-89991db5287a None admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:54.871386 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5930ca17-a5c0-48e9-af3e-89991db5287a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:17:54.871686 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5930ca17-a5c0-48e9-af3e-89991db5287a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:54.873046 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5930ca17-a5c0-48e9-af3e-89991db5287a None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:54.874395 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-ecdfe5dc-dbea-443c-9722-2fddd0e83a78 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 17:17:54.874395 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:17:54.874395 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:17:54.874395 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:17:54.874395 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:17:54.874395 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:17:54.874395 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:17:54.874395 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:17:54.874395 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:17:54.874395 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:17:54.874395 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:17:54.874395 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:17:54.874395 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:17:54.874395 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:17:54.874395 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:17:54.874395 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:17:54.875213 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:17:54.875213 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:17:54.875213 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:17:54.875213 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:17:54.875213 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:17:54.875213 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:17:54.875213 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:17:54.875213 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:17:54.875213 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:17:54.875213 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:17:54.875213 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:17:54.875213 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:17:54.875213 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:17:54.875213 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:17:54.875213 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:17:54.875213 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:17:54.875999 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:17:54.875999 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:17:54.875999 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:17:54.875999 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:17:54.875999 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:17:54.875999 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:17:54.875999 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:17:54.875999 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:17:54.875999 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:17:54.875999 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:17:54.875999 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 17:17:54.875999 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:17:54.881270 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 180/369] 10.222.0.102 () {56 vars in 1104 bytes} [Thu Sep 1 17:17:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 372 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:17:54.882168 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdfe5dc-dbea-443c-9722-2fddd0e83a78 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:54.882489 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 190/370] 10.222.0.102 () {56 vars in 1104 bytes} [Thu Sep 1 17:17:54 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:54.883437 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5930ca17-a5c0-48e9-af3e-89991db5287a None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:54.958282 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:54.995246 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:55.000869 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5930ca17-a5c0-48e9-af3e-89991db5287a None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:55.040188 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361ed904-9f73-490f-a05f-4af647772374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:55.040891 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361ed904-9f73-490f-a05f-4af647772374 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:55.041174 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361ed904-9f73-490f-a05f-4af647772374 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:55.041474 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361ed904-9f73-490f-a05f-4af647772374 None admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:55.042823 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361ed904-9f73-490f-a05f-4af647772374 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:17:55.043147 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361ed904-9f73-490f-a05f-4af647772374 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:55.044766 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361ed904-9f73-490f-a05f-4af647772374 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:55.054563 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 191/371] 10.222.0.102 () {56 vars in 1106 bytes} [Thu Sep 1 17:17:54 2022] GET /identity/v3/roles?name=reader => generated 343 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:55.055448 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361ed904-9f73-490f-a05f-4af647772374 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:55.168663 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361ed904-9f73-490f-a05f-4af647772374 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:55.183296 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdfe5dc-dbea-443c-9722-2fddd0e83a78 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:55.205644 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4478ba8c-24ba-4bc1-bd2c-fbd157b084a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:55.206254 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4478ba8c-24ba-4bc1-bd2c-fbd157b084a0 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:55.206535 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4478ba8c-24ba-4bc1-bd2c-fbd157b084a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:55.206810 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4478ba8c-24ba-4bc1-bd2c-fbd157b084a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:55.225949 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:55.284422 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4478ba8c-24ba-4bc1-bd2c-fbd157b084a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:55.286104 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4478ba8c-24ba-4bc1-bd2c-fbd157b084a0 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:55.326007 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54d6d7e-e885-47f1-8af0-b295e6b44dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:55.327230 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54d6d7e-e885-47f1-8af0-b295e6b44dfa None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:55.327656 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54d6d7e-e885-47f1-8af0-b295e6b44dfa None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:55.328096 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54d6d7e-e885-47f1-8af0-b295e6b44dfa None admin] PATH_INFO: `/v3/projects/3325fd0bf7e2439aa626201e2ae25c6b` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:55.329419 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54d6d7e-e885-47f1-8af0-b295e6b44dfa None admin] RBAC: Authorizing `identity:get_project(project_id=3325fd0bf7e2439aa626201e2ae25c6b)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:55.331021 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54d6d7e-e885-47f1-8af0-b295e6b44dfa None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:55.333388 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 181/372] 10.222.0.102 () {56 vars in 1191 bytes} [Thu Sep 1 17:17:54 2022] GET /identity/v3/projects/3325fd0bf7e2439aa626201e2ae25c6b => generated 303 bytes in 452 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:55.334360 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54d6d7e-e885-47f1-8af0-b295e6b44dfa None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:55.393072 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 192/373] 10.222.0.102 () {58 vars in 1287 bytes} [Thu Sep 1 17:17:55 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 17:17:55.393965 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4478ba8c-24ba-4bc1-bd2c-fbd157b084a0 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:55.445709 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:55.539018 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4478ba8c-24ba-4bc1-bd2c-fbd157b084a0 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:55.581344 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54d6d7e-e885-47f1-8af0-b295e6b44dfa None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:55.591680 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac74268-52c1-4a9e-8329-cefea64dc395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:55.592560 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac74268-52c1-4a9e-8329-cefea64dc395 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:55.592984 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac74268-52c1-4a9e-8329-cefea64dc395 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:55.593423 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac74268-52c1-4a9e-8329-cefea64dc395 None admin] PATH_INFO: `/v3/groups/300c8dafaf184bf59b61b6f89bdcf7de` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:55.594936 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac74268-52c1-4a9e-8329-cefea64dc395 None admin] RBAC: Authorizing `identity:get_group(group_id=300c8dafaf184bf59b61b6f89bdcf7de)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:55.596526 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac74268-52c1-4a9e-8329-cefea64dc395 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:55.598112 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:55.598502 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 193/374] 10.222.0.102 () {56 vars in 1185 bytes} [Thu Sep 1 17:17:55 2022] GET /identity/v3/groups/300c8dafaf184bf59b61b6f89bdcf7de => generated 227 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:55.599469 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac74268-52c1-4a9e-8329-cefea64dc395 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:55.628352 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5696311-c57a-4970-8870-329dafd412dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:55.628997 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5696311-c57a-4970-8870-329dafd412dd None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:55.629873 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5696311-c57a-4970-8870-329dafd412dd None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:55.630174 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5696311-c57a-4970-8870-329dafd412dd None admin] PATH_INFO: `/v3/users/6b2665ec5f804ecfa24c8d986daea4ad` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:55.631335 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5696311-c57a-4970-8870-329dafd412dd None admin] RBAC: Authorizing `identity:get_user(user_id=6b2665ec5f804ecfa24c8d986daea4ad)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:55.632753 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5696311-c57a-4970-8870-329dafd412dd None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:55.636513 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 182/375] 10.222.0.102 () {56 vars in 1182 bytes} [Thu Sep 1 17:17:55 2022] GET /identity/v3/users/6b2665ec5f804ecfa24c8d986daea4ad => generated 294 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:55.637318 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5696311-c57a-4970-8870-329dafd412dd None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:55.769996 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac74268-52c1-4a9e-8329-cefea64dc395 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:55.789056 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:55.812539 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8e73a6-370c-4839-b1e5-89f4e1d963fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:55.813541 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8e73a6-370c-4839-b1e5-89f4e1d963fe None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:55.813673 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5696311-c57a-4970-8870-329dafd412dd None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:55.814605 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8e73a6-370c-4839-b1e5-89f4e1d963fe None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:55.814605 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8e73a6-370c-4839-b1e5-89f4e1d963fe None admin] PATH_INFO: `/v3/projects/ace9b3845813477eb192c158600d0beb` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:55.819590 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8e73a6-370c-4839-b1e5-89f4e1d963fe None admin] RBAC: Authorizing `identity:get_project(project_id=ace9b3845813477eb192c158600d0beb)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:55.819590 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8e73a6-370c-4839-b1e5-89f4e1d963fe None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:55.820054 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 194/376] 10.222.0.102 () {56 vars in 1191 bytes} [Thu Sep 1 17:17:55 2022] GET /identity/v3/projects/ace9b3845813477eb192c158600d0beb => generated 345 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:55.821045 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8e73a6-370c-4839-b1e5-89f4e1d963fe None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:55.853143 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d8f14d-14fa-41a9-b71c-ad23318e886f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:55.853706 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d8f14d-14fa-41a9-b71c-ad23318e886f None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:55.853984 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d8f14d-14fa-41a9-b71c-ad23318e886f None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:55.854255 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d8f14d-14fa-41a9-b71c-ad23318e886f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:55.854986 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d8f14d-14fa-41a9-b71c-ad23318e886f None admin] RBAC: Adding query filter params (group.id=300c8dafaf184bf59b61b6f89bdcf7de, role.id=abef78c185c74c16a2c7e8a170583f9f, scope.project.id=3325fd0bf7e2439aa626201e2ae25c6b) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:17:55.855295 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d8f14d-14fa-41a9-b71c-ad23318e886f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:55.860219 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d8f14d-14fa-41a9-b71c-ad23318e886f None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:55.863045 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 183/377] 10.222.0.102 () {56 vars in 1381 bytes} [Thu Sep 1 17:17:55 2022] GET /identity/v3/role_assignments?group.id=300c8dafaf184bf59b61b6f89bdcf7de&scope.project.id=3325fd0bf7e2439aa626201e2ae25c6b&role.id=abef78c185c74c16a2c7e8a170583f9f => generated 261 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:55.864575 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d8f14d-14fa-41a9-b71c-ad23318e886f None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:55.912247 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:55.937388 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8e73a6-370c-4839-b1e5-89f4e1d963fe None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:55.975630 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bfa0b7-9404-45f9-82b2-35a0c5eda8d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:55.976253 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bfa0b7-9404-45f9-82b2-35a0c5eda8d9 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:55.976534 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bfa0b7-9404-45f9-82b2-35a0c5eda8d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:55.976809 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bfa0b7-9404-45f9-82b2-35a0c5eda8d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:55.977846 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bfa0b7-9404-45f9-82b2-35a0c5eda8d9 None admin] RBAC: Adding query filter params (role.id=ab816d03be024513884447335f2de35a, user.id=6b2665ec5f804ecfa24c8d986daea4ad, scope.system=all) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:17:55.978162 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bfa0b7-9404-45f9-82b2-35a0c5eda8d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:55.979382 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d8f14d-14fa-41a9-b71c-ad23318e886f None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:55.979725 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bfa0b7-9404-45f9-82b2-35a0c5eda8d9 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:55.997703 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 195/378] 10.222.0.102 () {56 vars in 1313 bytes} [Thu Sep 1 17:17:55 2022] GET /identity/v3/role_assignments?user.id=6b2665ec5f804ecfa24c8d986daea4ad&scope.system=all&role.id=ab816d03be024513884447335f2de35a => generated 512 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:56.019941 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109b9f1d-1606-4679-b21a-7329baaebb40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:56.021101 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109b9f1d-1606-4679-b21a-7329baaebb40 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:56.021412 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109b9f1d-1606-4679-b21a-7329baaebb40 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:56.021686 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109b9f1d-1606-4679-b21a-7329baaebb40 None admin] PATH_INFO: `/v3/groups/acbd924b0e244da4ae0f704418cd4e68` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:56.029640 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109b9f1d-1606-4679-b21a-7329baaebb40 None admin] RBAC: Authorizing `identity:get_group(group_id=acbd924b0e244da4ae0f704418cd4e68)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:56.031158 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109b9f1d-1606-4679-b21a-7329baaebb40 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:56.033221 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 184/379] 10.222.0.102 () {56 vars in 1185 bytes} [Thu Sep 1 17:17:55 2022] GET /identity/v3/groups/acbd924b0e244da4ae0f704418cd4e68 => generated 230 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:56.039109 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bfa0b7-9404-45f9-82b2-35a0c5eda8d9 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:56.098437 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:56.172395 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bfa0b7-9404-45f9-82b2-35a0c5eda8d9 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:56.217607 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3256f7e3-6f7a-4a0a-9985-5afe12aaf283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:56.218618 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3256f7e3-6f7a-4a0a-9985-5afe12aaf283 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:56.219157 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3256f7e3-6f7a-4a0a-9985-5afe12aaf283 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:56.219667 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3256f7e3-6f7a-4a0a-9985-5afe12aaf283 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:56.220604 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3256f7e3-6f7a-4a0a-9985-5afe12aaf283 None admin] RBAC: Adding query filter params (group.id=acbd924b0e244da4ae0f704418cd4e68, role.id=36577a6f46d642d4b81805ede9872490, scope.project.id=ace9b3845813477eb192c158600d0beb) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:17:56.221122 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3256f7e3-6f7a-4a0a-9985-5afe12aaf283 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:56.222790 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3256f7e3-6f7a-4a0a-9985-5afe12aaf283 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:56.230109 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 196/380] 10.222.0.102 () {56 vars in 1381 bytes} [Thu Sep 1 17:17:56 2022] GET /identity/v3/role_assignments?group.id=acbd924b0e244da4ae0f704418cd4e68&scope.project.id=ace9b3845813477eb192c158600d0beb&role.id=36577a6f46d642d4b81805ede9872490 => generated 261 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:17:56.268661 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7196fc70-1c06-4f3f-980b-1970d1048805 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:56.269296 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7196fc70-1c06-4f3f-980b-1970d1048805 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:56.269822 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7196fc70-1c06-4f3f-980b-1970d1048805 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:56.272039 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 185/381] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:17:56 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:17:56.282258 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93accbc-0ea7-497d-9e72-d90aea22bed4 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:56.282897 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93accbc-0ea7-497d-9e72-d90aea22bed4 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:56.283675 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93accbc-0ea7-497d-9e72-d90aea22bed4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:56.309620 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-b93accbc-0ea7-497d-9e72-d90aea22bed4 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:17:56.310862 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-b93accbc-0ea7-497d-9e72-d90aea22bed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:17:56.358433 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34c407b-0719-4a8a-9689-915c674279c1 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:56.358971 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34c407b-0719-4a8a-9689-915c674279c1 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:56.358971 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34c407b-0719-4a8a-9689-915c674279c1 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:56.360758 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 186/382] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:17:56 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:17:56.367548 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d16507-dd27-4b76-bd44-8e5ad0774616 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:56.367851 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d16507-dd27-4b76-bd44-8e5ad0774616 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:56.368112 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d16507-dd27-4b76-bd44-8e5ad0774616 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:56.391543 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-70d16507-dd27-4b76-bd44-8e5ad0774616 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:17:56.393816 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-70d16507-dd27-4b76-bd44-8e5ad0774616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:17:56.424120 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 197/383] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:17:56 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:17:56.491618 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 187/384] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:17:56 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:17:56.733913 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93accbc-0ea7-497d-9e72-d90aea22bed4 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:56.768930 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:56.858917 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93accbc-0ea7-497d-9e72-d90aea22bed4 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:56.896589 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d09b7ca-f0ff-49b3-b520-715b8e0e4433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:56.897602 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d09b7ca-f0ff-49b3-b520-715b8e0e4433 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:56.898115 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d09b7ca-f0ff-49b3-b520-715b8e0e4433 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:56.898644 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d09b7ca-f0ff-49b3-b520-715b8e0e4433 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:56.909575 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d09b7ca-f0ff-49b3-b520-715b8e0e4433 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:17:56.915470 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-5d09b7ca-f0ff-49b3-b520-715b8e0e4433 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:17:56.915470 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:17:56.915470 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:17:56.915470 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:17:56.915470 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:17:56.915470 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:17:56.915470 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:17:56.915470 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:17:56.915470 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:17:56.915470 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:17:56.915470 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:17:56.915470 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:17:56.915470 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:17:56.915470 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:17:56.915470 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:17:56.915470 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:17:56.916350 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:17:56.916350 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:17:56.916350 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:17:56.916350 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:17:56.916350 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:17:56.916350 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:17:56.916350 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:17:56.916350 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:17:56.916350 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:17:56.916350 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:17:56.916350 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:17:56.916350 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:17:56.916350 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:17:56.916350 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:17:56.916350 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:17:56.916350 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:17:56.917155 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:17:56.917155 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:17:56.917155 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:17:56.917155 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:17:56.917155 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:17:56.917155 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:17:56.917155 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:17:56.917155 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:17:56.917155 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:17:56.917155 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:17:56.917155 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:17:56.917155 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:17:56.920885 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 198/385] 10.222.0.102 () {56 vars in 1104 bytes} [Thu Sep 1 17:17:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:17:56.927451 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d16507-dd27-4b76-bd44-8e5ad0774616 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:56.970047 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:57.057454 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d16507-dd27-4b76-bd44-8e5ad0774616 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:57.068400 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d09b7ca-f0ff-49b3-b520-715b8e0e4433 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:57.096619 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee501588-7278-4400-8a3d-82a6df2f750d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:57.097514 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee501588-7278-4400-8a3d-82a6df2f750d None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:57.097649 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee501588-7278-4400-8a3d-82a6df2f750d None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:57.097901 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee501588-7278-4400-8a3d-82a6df2f750d None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:57.099630 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee501588-7278-4400-8a3d-82a6df2f750d None admin] RBAC: Adding query filter params (name=member) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:17:57.099934 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee501588-7278-4400-8a3d-82a6df2f750d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:57.101353 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee501588-7278-4400-8a3d-82a6df2f750d None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:57.111350 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 188/386] 10.222.0.102 () {56 vars in 1106 bytes} [Thu Sep 1 17:17:56 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:57.112600 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035e39c9-5d02-4192-b5a9-7ad70c5efe20 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:57.112888 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035e39c9-5d02-4192-b5a9-7ad70c5efe20 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:57.113161 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035e39c9-5d02-4192-b5a9-7ad70c5efe20 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:57.119119 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 189/387] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:17:57 2022] GET /identity => generated 271 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:17:57.120324 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035e39c9-5d02-4192-b5a9-7ad70c5efe20 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:57.193763 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:57.203964 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d09b7ca-f0ff-49b3-b520-715b8e0e4433 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:57.250038 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0e9366-1288-4125-bd6c-641b1f91e4bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:57.251038 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0e9366-1288-4125-bd6c-641b1f91e4bf None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:57.251847 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0e9366-1288-4125-bd6c-641b1f91e4bf None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:57.252301 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0e9366-1288-4125-bd6c-641b1f91e4bf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:57.259974 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:57.266194 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035e39c9-5d02-4192-b5a9-7ad70c5efe20 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:57.270538 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0e9366-1288-4125-bd6c-641b1f91e4bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:17:57.275690 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-4f0e9366-1288-4125-bd6c-641b1f91e4bf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 17:17:57.275690 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:17:57.275690 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:17:57.275690 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:17:57.275690 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:17:57.275690 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:17:57.275690 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:17:57.275690 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:17:57.275690 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:17:57.275690 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:17:57.275690 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:17:57.275690 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:17:57.275690 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:17:57.275690 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:17:57.275690 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:17:57.275690 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:17:57.276573 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:17:57.276573 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:17:57.276573 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:17:57.276573 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:17:57.276573 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:17:57.276573 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:17:57.276573 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:17:57.276573 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:17:57.276573 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:17:57.276573 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:17:57.276573 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:17:57.276573 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:17:57.276573 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:17:57.276573 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:17:57.276573 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:17:57.276573 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:17:57.277419 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:17:57.277419 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:17:57.277419 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:17:57.277419 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:17:57.277419 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:17:57.277419 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:17:57.277419 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:17:57.277419 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:17:57.277419 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:17:57.277419 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:17:57.277419 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 17:17:57.277419 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:17:57.280111 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 199/388] 10.222.0.102 () {56 vars in 1119 bytes} [Thu Sep 1 17:17:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:17:57.281260 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ff0120-45cf-4fa1-92a2-99c30fb653e5 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:57.281579 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ff0120-45cf-4fa1-92a2-99c30fb653e5 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:57.283534 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ff0120-45cf-4fa1-92a2-99c30fb653e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:57.310406 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-66ff0120-45cf-4fa1-92a2-99c30fb653e5 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:17:57.312374 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-66ff0120-45cf-4fa1-92a2-99c30fb653e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:17:57.314108 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6e7c08-78c2-4c79-a8dd-0fa2bd17bbea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:57.314751 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6e7c08-78c2-4c79-a8dd-0fa2bd17bbea None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:57.315028 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6e7c08-78c2-4c79-a8dd-0fa2bd17bbea None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:57.315303 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6e7c08-78c2-4c79-a8dd-0fa2bd17bbea None admin] PATH_INFO: `/v3/groups/300c8dafaf184bf59b61b6f89bdcf7de` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:57.316415 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6e7c08-78c2-4c79-a8dd-0fa2bd17bbea None admin] RBAC: Authorizing `identity:get_group(group_id=300c8dafaf184bf59b61b6f89bdcf7de)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:57.318446 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6e7c08-78c2-4c79-a8dd-0fa2bd17bbea None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:57.324603 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 190/389] 10.222.0.102 () {56 vars in 1185 bytes} [Thu Sep 1 17:17:57 2022] GET /identity/v3/groups/300c8dafaf184bf59b61b6f89bdcf7de => generated 227 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:57.325519 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6e7c08-78c2-4c79-a8dd-0fa2bd17bbea None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:57.401472 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 200/390] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:17:57 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:17:57.402660 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ff0120-45cf-4fa1-92a2-99c30fb653e5 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:57.462307 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6e7c08-78c2-4c79-a8dd-0fa2bd17bbea None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:57.512700 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fae532-36f8-4823-ba3c-bae28242d2a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:57.514022 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fae532-36f8-4823-ba3c-bae28242d2a2 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:57.514535 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fae532-36f8-4823-ba3c-bae28242d2a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:57.515339 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fae532-36f8-4823-ba3c-bae28242d2a2 None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:57.516465 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fae532-36f8-4823-ba3c-bae28242d2a2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:17:57.516961 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fae532-36f8-4823-ba3c-bae28242d2a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:57.519253 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fae532-36f8-4823-ba3c-bae28242d2a2 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:57.521907 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:57.532758 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 191/391] 10.222.0.102 () {56 vars in 1116 bytes} [Thu Sep 1 17:17:57 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:57.539234 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fae532-36f8-4823-ba3c-bae28242d2a2 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:57.549518 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ff0120-45cf-4fa1-92a2-99c30fb653e5 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:57.576741 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:57.600406 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38235ffa-7fe2-40e6-92b3-f6c41448d238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:57.601444 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38235ffa-7fe2-40e6-92b3-f6c41448d238 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:57.601918 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38235ffa-7fe2-40e6-92b3-f6c41448d238 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:57.602394 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38235ffa-7fe2-40e6-92b3-f6c41448d238 None admin] PATH_INFO: `/v3/projects/8e88c67b942f45108c95a833547f9bc8` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:57.604187 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38235ffa-7fe2-40e6-92b3-f6c41448d238 None admin] RBAC: Authorizing `identity:get_project(project_id=8e88c67b942f45108c95a833547f9bc8)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:57.606244 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38235ffa-7fe2-40e6-92b3-f6c41448d238 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:57.608572 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 201/392] 10.222.0.102 () {56 vars in 1191 bytes} [Thu Sep 1 17:17:57 2022] GET /identity/v3/projects/8e88c67b942f45108c95a833547f9bc8 => generated 299 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:57.616362 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38235ffa-7fe2-40e6-92b3-f6c41448d238 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:57.706012 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fae532-36f8-4823-ba3c-bae28242d2a2 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:57.774730 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56b95aa-19a2-4b9e-a688-45f6f9270852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:57.775377 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56b95aa-19a2-4b9e-a688-45f6f9270852 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:57.775655 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56b95aa-19a2-4b9e-a688-45f6f9270852 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:57.775932 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56b95aa-19a2-4b9e-a688-45f6f9270852 None admin] PATH_INFO: `/v3/groups/300c8dafaf184bf59b61b6f89bdcf7de` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:57.777190 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56b95aa-19a2-4b9e-a688-45f6f9270852 None admin] RBAC: Authorizing `identity:get_group(group_id=300c8dafaf184bf59b61b6f89bdcf7de)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:57.778682 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56b95aa-19a2-4b9e-a688-45f6f9270852 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:57.784858 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 192/393] 10.222.0.102 () {56 vars in 1185 bytes} [Thu Sep 1 17:17:57 2022] GET /identity/v3/groups/300c8dafaf184bf59b61b6f89bdcf7de => generated 227 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:57.786859 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56b95aa-19a2-4b9e-a688-45f6f9270852 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:57.831639 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38235ffa-7fe2-40e6-92b3-f6c41448d238 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:57.892840 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e425d821-08cc-4e16-9bc3-5d301442028a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:57.893556 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e425d821-08cc-4e16-9bc3-5d301442028a None admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:57.893837 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e425d821-08cc-4e16-9bc3-5d301442028a None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:57.894115 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e425d821-08cc-4e16-9bc3-5d301442028a None admin] PATH_INFO: `/v3/projects/8e88c67b942f45108c95a833547f9bc8/groups/300c8dafaf184bf59b61b6f89bdcf7de/roles/b75cb360925e4ba5a65f14965c103ffd` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:57.896547 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e425d821-08cc-4e16-9bc3-5d301442028a None admin] RBAC: Authorizing `identity:create_grant(project_id=8e88c67b942f45108c95a833547f9bc8, group_id=300c8dafaf184bf59b61b6f89bdcf7de, role_id=b75cb360925e4ba5a65f14965c103ffd)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:57.898119 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e425d821-08cc-4e16-9bc3-5d301442028a None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:57.912572 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:57.918805 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 202/394] 10.222.0.102 () {58 vars in 1447 bytes} [Thu Sep 1 17:17:57 2022] PUT /identity/v3/projects/8e88c67b942f45108c95a833547f9bc8/groups/300c8dafaf184bf59b61b6f89bdcf7de/roles/b75cb360925e4ba5a65f14965c103ffd => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:17:57.923406 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e425d821-08cc-4e16-9bc3-5d301442028a None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:57.929000 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56b95aa-19a2-4b9e-a688-45f6f9270852 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:57.965732 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:58.007174 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9a747a-d134-476c-80a9-de84d847b250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:58.008249 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9a747a-d134-476c-80a9-de84d847b250 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:58.008793 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9a747a-d134-476c-80a9-de84d847b250 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:58.009273 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9a747a-d134-476c-80a9-de84d847b250 None admin] PATH_INFO: `/v3/projects/8e88c67b942f45108c95a833547f9bc8` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:58.011849 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9a747a-d134-476c-80a9-de84d847b250 None admin] RBAC: Authorizing `identity:get_project(project_id=8e88c67b942f45108c95a833547f9bc8)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:58.013575 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9a747a-d134-476c-80a9-de84d847b250 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:58.018062 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 193/395] 10.222.0.102 () {56 vars in 1191 bytes} [Thu Sep 1 17:17:57 2022] GET /identity/v3/projects/8e88c67b942f45108c95a833547f9bc8 => generated 299 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:58.023483 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9a747a-d134-476c-80a9-de84d847b250 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:58.154628 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e425d821-08cc-4e16-9bc3-5d301442028a None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:58.159056 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9a747a-d134-476c-80a9-de84d847b250 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:58.205262 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bded16-8356-4724-9ef8-cdd16b4b9ac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:58.206337 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bded16-8356-4724-9ef8-cdd16b4b9ac1 None admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:58.206814 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bded16-8356-4724-9ef8-cdd16b4b9ac1 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:58.207281 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bded16-8356-4724-9ef8-cdd16b4b9ac1 None admin] PATH_INFO: `/v3/projects/8e88c67b942f45108c95a833547f9bc8/groups/300c8dafaf184bf59b61b6f89bdcf7de/roles/abef78c185c74c16a2c7e8a170583f9f` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:58.210233 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bded16-8356-4724-9ef8-cdd16b4b9ac1 None admin] RBAC: Authorizing `identity:create_grant(project_id=8e88c67b942f45108c95a833547f9bc8, group_id=300c8dafaf184bf59b61b6f89bdcf7de, role_id=abef78c185c74c16a2c7e8a170583f9f)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:58.211842 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbf6e05-8b3e-4abf-a43e-4af54b7dc82c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:58.212467 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bded16-8356-4724-9ef8-cdd16b4b9ac1 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:58.212576 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbf6e05-8b3e-4abf-a43e-4af54b7dc82c None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:58.212749 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbf6e05-8b3e-4abf-a43e-4af54b7dc82c None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:58.213021 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbf6e05-8b3e-4abf-a43e-4af54b7dc82c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:58.226127 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbf6e05-8b3e-4abf-a43e-4af54b7dc82c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:17:58.230461 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 194/396] 10.222.0.102 () {58 vars in 1447 bytes} [Thu Sep 1 17:17:58 2022] PUT /identity/v3/projects/8e88c67b942f45108c95a833547f9bc8/groups/300c8dafaf184bf59b61b6f89bdcf7de/roles/abef78c185c74c16a2c7e8a170583f9f => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:17:58.240002 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-7bbf6e05-8b3e-4abf-a43e-4af54b7dc82c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:17:58.240002 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:17:58.240002 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:17:58.240002 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:17:58.240002 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:17:58.240002 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:17:58.240002 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:17:58.240002 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:17:58.240002 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:17:58.240002 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:17:58.240002 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:17:58.240002 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:17:58.240002 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:17:58.240002 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:17:58.240002 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:17:58.240002 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:17:58.240954 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:17:58.240954 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:17:58.240954 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:17:58.240954 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:17:58.240954 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:17:58.240954 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:17:58.240954 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:17:58.240954 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:17:58.240954 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:17:58.240954 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:17:58.240954 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:17:58.240954 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:17:58.240954 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:17:58.240954 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:17:58.240954 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:17:58.240954 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:17:58.241798 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:17:58.241798 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:17:58.241798 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:17:58.241798 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:17:58.241798 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:17:58.241798 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:17:58.241798 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:17:58.241798 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:17:58.241798 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:17:58.241798 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:17:58.241798 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:17:58.241798 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:17:58.244464 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 203/397] 10.222.0.102 () {56 vars in 1104 bytes} [Thu Sep 1 17:17:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:17:58.250585 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bded16-8356-4724-9ef8-cdd16b4b9ac1 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:58.293791 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:58.386490 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bded16-8356-4724-9ef8-cdd16b4b9ac1 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:58.427490 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82708655-2398-464a-844a-0e87416185b4 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:58.427967 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82708655-2398-464a-844a-0e87416185b4 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:58.427967 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82708655-2398-464a-844a-0e87416185b4 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:58.432064 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 204/398] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:17:58 2022] GET /identity => generated 271 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:17:58.438364 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c176bbb0-0a6a-43a2-8b5b-895f61f544ab None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:58.438654 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c176bbb0-0a6a-43a2-8b5b-895f61f544ab None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:58.439009 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c176bbb0-0a6a-43a2-8b5b-895f61f544ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:58.439493 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e6601b-3ca3-4036-aa05-8e851bfdc28c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:58.440171 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e6601b-3ca3-4036-aa05-8e851bfdc28c None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:58.440450 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e6601b-3ca3-4036-aa05-8e851bfdc28c None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:58.440721 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e6601b-3ca3-4036-aa05-8e851bfdc28c None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:58.441424 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e6601b-3ca3-4036-aa05-8e851bfdc28c None admin] RBAC: Adding query filter params (name=member) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:17:58.441798 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e6601b-3ca3-4036-aa05-8e851bfdc28c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:58.443907 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e6601b-3ca3-4036-aa05-8e851bfdc28c None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:58.454644 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 195/399] 10.222.0.102 () {56 vars in 1106 bytes} [Thu Sep 1 17:17:58 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:58.475113 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e6601b-3ca3-4036-aa05-8e851bfdc28c None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:58.480049 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-c176bbb0-0a6a-43a2-8b5b-895f61f544ab None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:17:58.481284 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-c176bbb0-0a6a-43a2-8b5b-895f61f544ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:17:58.569449 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 205/400] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:17:58 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:17:58.579539 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:58.603848 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e6601b-3ca3-4036-aa05-8e851bfdc28c None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:58.641063 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591300e1-5298-47db-b34b-b7f01ba34ee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:58.642191 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591300e1-5298-47db-b34b-b7f01ba34ee7 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:58.642695 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591300e1-5298-47db-b34b-b7f01ba34ee7 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:58.643174 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591300e1-5298-47db-b34b-b7f01ba34ee7 None admin] PATH_INFO: `/v3/groups/300c8dafaf184bf59b61b6f89bdcf7de` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:58.644539 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591300e1-5298-47db-b34b-b7f01ba34ee7 None admin] RBAC: Authorizing `identity:get_group(group_id=300c8dafaf184bf59b61b6f89bdcf7de)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:58.646244 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591300e1-5298-47db-b34b-b7f01ba34ee7 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:58.648381 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 196/401] 10.222.0.102 () {56 vars in 1185 bytes} [Thu Sep 1 17:17:58 2022] GET /identity/v3/groups/300c8dafaf184bf59b61b6f89bdcf7de => generated 227 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:58.653630 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c176bbb0-0a6a-43a2-8b5b-895f61f544ab None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:58.656423 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:58.817336 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c176bbb0-0a6a-43a2-8b5b-895f61f544ab None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:58.856767 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463f560e-ca1d-4da7-bdec-1ca69e6b127f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:58.857446 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463f560e-ca1d-4da7-bdec-1ca69e6b127f None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:58.857730 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463f560e-ca1d-4da7-bdec-1ca69e6b127f None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:58.858065 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463f560e-ca1d-4da7-bdec-1ca69e6b127f None admin] PATH_INFO: `/v3/projects/3325fd0bf7e2439aa626201e2ae25c6b` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:58.859281 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463f560e-ca1d-4da7-bdec-1ca69e6b127f None admin] RBAC: Authorizing `identity:get_project(project_id=3325fd0bf7e2439aa626201e2ae25c6b)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:58.860776 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463f560e-ca1d-4da7-bdec-1ca69e6b127f None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:58.863476 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 206/402] 10.222.0.102 () {56 vars in 1191 bytes} [Thu Sep 1 17:17:58 2022] GET /identity/v3/projects/3325fd0bf7e2439aa626201e2ae25c6b => generated 303 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:58.869546 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591300e1-5298-47db-b34b-b7f01ba34ee7 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:58.891671 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463f560e-ca1d-4da7-bdec-1ca69e6b127f None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:59.005681 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:59.016550 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591300e1-5298-47db-b34b-b7f01ba34ee7 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:59.022372 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463f560e-ca1d-4da7-bdec-1ca69e6b127f None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:59.057688 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a334513-b64a-48ea-8ceb-717b27656bba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:59.058360 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a334513-b64a-48ea-8ceb-717b27656bba None admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:59.058641 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a334513-b64a-48ea-8ceb-717b27656bba None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:59.058920 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a334513-b64a-48ea-8ceb-717b27656bba None admin] PATH_INFO: `/v3/projects/3325fd0bf7e2439aa626201e2ae25c6b/groups/300c8dafaf184bf59b61b6f89bdcf7de/roles/b75cb360925e4ba5a65f14965c103ffd` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:59.060802 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a334513-b64a-48ea-8ceb-717b27656bba None admin] RBAC: Authorizing `identity:create_grant(project_id=3325fd0bf7e2439aa626201e2ae25c6b, group_id=300c8dafaf184bf59b61b6f89bdcf7de, role_id=b75cb360925e4ba5a65f14965c103ffd)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:59.060926 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfb186e-64cc-4ccb-8a35-4c1ccbc2e74d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:59.061518 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb186e-64cc-4ccb-8a35-4c1ccbc2e74d None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:59.061848 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb186e-64cc-4ccb-8a35-4c1ccbc2e74d None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:59.062067 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb186e-64cc-4ccb-8a35-4c1ccbc2e74d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:59.062364 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a334513-b64a-48ea-8ceb-717b27656bba None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:59.075212 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfb186e-64cc-4ccb-8a35-4c1ccbc2e74d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:17:59.080335 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 197/403] 10.222.0.102 () {58 vars in 1447 bytes} [Thu Sep 1 17:17:58 2022] PUT /identity/v3/projects/3325fd0bf7e2439aa626201e2ae25c6b/groups/300c8dafaf184bf59b61b6f89bdcf7de/roles/b75cb360925e4ba5a65f14965c103ffd => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:17:59.080754 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-4dfb186e-64cc-4ccb-8a35-4c1ccbc2e74d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 17:17:59.080754 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:17:59.080754 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:17:59.080754 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:17:59.080754 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:17:59.080754 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:17:59.080754 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:17:59.080754 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:17:59.080754 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:17:59.080754 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:17:59.080754 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:17:59.080754 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:17:59.080754 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:17:59.080754 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:17:59.080754 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:17:59.080754 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:17:59.081700 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:17:59.081700 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:17:59.081700 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:17:59.081700 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:17:59.081700 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:17:59.081700 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:17:59.081700 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:17:59.081700 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:17:59.081700 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:17:59.081700 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:17:59.081700 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:17:59.081700 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:17:59.081700 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:17:59.081700 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:17:59.081700 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:17:59.081700 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:17:59.083042 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:17:59.083042 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:17:59.083042 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:17:59.083042 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:17:59.083042 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:17:59.083042 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:17:59.083042 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:17:59.083042 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:17:59.083042 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:17:59.083042 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:17:59.083042 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 17:17:59.083042 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:17:59.083670 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63c17d2-8dd0-4987-8a9a-d8cbd495d4ea None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:59.083752 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 207/404] 10.222.0.102 () {56 vars in 1119 bytes} [Thu Sep 1 17:17:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:17:59.083824 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63c17d2-8dd0-4987-8a9a-d8cbd495d4ea None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:59.083824 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63c17d2-8dd0-4987-8a9a-d8cbd495d4ea None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:59.089186 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 198/405] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:17:59 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:17:59.089545 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfb186e-64cc-4ccb-8a35-4c1ccbc2e74d None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:59.091032 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42ad3aa-cd0e-4e5f-bc1b-55d914453282 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:59.091472 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42ad3aa-cd0e-4e5f-bc1b-55d914453282 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:59.091781 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42ad3aa-cd0e-4e5f-bc1b-55d914453282 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:59.129555 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-c42ad3aa-cd0e-4e5f-bc1b-55d914453282 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:17:59.130813 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-c42ad3aa-cd0e-4e5f-bc1b-55d914453282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:17:59.150251 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:59.285144 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 199/406] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:17:59 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 17:17:59.294039 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfb186e-64cc-4ccb-8a35-4c1ccbc2e74d None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:59.331041 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:59.338116 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6952da9-a8a8-492c-8a40-7311d173e678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:59.338732 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6952da9-a8a8-492c-8a40-7311d173e678 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:59.339017 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6952da9-a8a8-492c-8a40-7311d173e678 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:59.339342 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6952da9-a8a8-492c-8a40-7311d173e678 None admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:59.340018 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6952da9-a8a8-492c-8a40-7311d173e678 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:17:59.340370 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6952da9-a8a8-492c-8a40-7311d173e678 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:59.341844 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6952da9-a8a8-492c-8a40-7311d173e678 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:59.351458 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 208/407] 10.222.0.102 () {56 vars in 1116 bytes} [Thu Sep 1 17:17:59 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:17:59.356978 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42ad3aa-cd0e-4e5f-bc1b-55d914453282 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:59.472292 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42ad3aa-cd0e-4e5f-bc1b-55d914453282 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:59.512496 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b62e12-e155-4b85-8675-9c130002a6c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:59.513148 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b62e12-e155-4b85-8675-9c130002a6c8 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:59.513435 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b62e12-e155-4b85-8675-9c130002a6c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:59.513711 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b62e12-e155-4b85-8675-9c130002a6c8 None admin] PATH_INFO: `/v3/groups/300c8dafaf184bf59b61b6f89bdcf7de` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:59.514829 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b62e12-e155-4b85-8675-9c130002a6c8 None admin] RBAC: Authorizing `identity:get_group(group_id=300c8dafaf184bf59b61b6f89bdcf7de)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:59.516278 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b62e12-e155-4b85-8675-9c130002a6c8 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:59.518317 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 200/408] 10.222.0.102 () {56 vars in 1185 bytes} [Thu Sep 1 17:17:59 2022] GET /identity/v3/groups/300c8dafaf184bf59b61b6f89bdcf7de => generated 227 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:17:59.524594 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6952da9-a8a8-492c-8a40-7311d173e678 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:59.598169 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b62e12-e155-4b85-8675-9c130002a6c8 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:59.628187 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:59.670028 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6952da9-a8a8-492c-8a40-7311d173e678 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:59.724686 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b96676-33af-4d63-9821-f9ee27c48c35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:59.725357 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b96676-33af-4d63-9821-f9ee27c48c35 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:59.725618 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b96676-33af-4d63-9821-f9ee27c48c35 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:59.725884 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b96676-33af-4d63-9821-f9ee27c48c35 None admin] PATH_INFO: `/v3/projects/3325fd0bf7e2439aa626201e2ae25c6b` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:59.728888 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b96676-33af-4d63-9821-f9ee27c48c35 None admin] RBAC: Authorizing `identity:get_project(project_id=3325fd0bf7e2439aa626201e2ae25c6b)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:17:59.730364 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b96676-33af-4d63-9821-f9ee27c48c35 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:17:59.741736 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 209/409] 10.222.0.102 () {56 vars in 1191 bytes} [Thu Sep 1 17:17:59 2022] GET /identity/v3/projects/3325fd0bf7e2439aa626201e2ae25c6b => generated 303 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:17:59.755629 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b96676-33af-4d63-9821-f9ee27c48c35 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:59.826290 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b62e12-e155-4b85-8675-9c130002a6c8 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:59.875541 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:17:59.886050 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86783da8-8bb4-4928-9abe-6c3058d7ed6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:59.886742 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86783da8-8bb4-4928-9abe-6c3058d7ed6b None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:59.887021 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86783da8-8bb4-4928-9abe-6c3058d7ed6b None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:59.887289 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86783da8-8bb4-4928-9abe-6c3058d7ed6b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:17:59.906596 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86783da8-8bb4-4928-9abe-6c3058d7ed6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:17:59.911507 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-86783da8-8bb4-4928-9abe-6c3058d7ed6b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:17:59.911507 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:17:59.911507 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:17:59.911507 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:17:59.911507 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:17:59.911507 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:17:59.911507 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:17:59.911507 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:17:59.911507 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:17:59.911507 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:17:59.911507 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:17:59.911507 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:17:59.911507 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:17:59.911507 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:17:59.911507 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:17:59.911507 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:17:59.912451 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:17:59.912451 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:17:59.912451 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:17:59.912451 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:17:59.912451 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:17:59.912451 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:17:59.912451 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:17:59.912451 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:17:59.912451 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:17:59.912451 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:17:59.912451 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:17:59.912451 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:17:59.912451 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:17:59.912451 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:17:59.912451 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:17:59.912451 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:17:59.913226 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:17:59.913226 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:17:59.913226 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:17:59.913226 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:17:59.913226 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:17:59.913226 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:17:59.913226 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:17:59.913226 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:17:59.913226 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:17:59.913226 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:17:59.913226 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:17:59.913226 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:17:59.913226 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 201/410] 10.222.0.102 () {56 vars in 1101 bytes} [Thu Sep 1 17:17:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:17:59.918804 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b96676-33af-4d63-9821-f9ee27c48c35 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:17:59.921006 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86783da8-8bb4-4928-9abe-6c3058d7ed6b None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:17:59.998636 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e7c551-34e9-4769-892d-f771de2c927d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:17:59.999304 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e7c551-34e9-4769-892d-f771de2c927d None admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:17:59.999580 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e7c551-34e9-4769-892d-f771de2c927d None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:17:59.999851 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e7c551-34e9-4769-892d-f771de2c927d None admin] PATH_INFO: `/v3/projects/3325fd0bf7e2439aa626201e2ae25c6b/groups/300c8dafaf184bf59b61b6f89bdcf7de/roles/abef78c185c74c16a2c7e8a170583f9f` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:00.002868 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:00.005411 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e7c551-34e9-4769-892d-f771de2c927d None admin] RBAC: Authorizing `identity:create_grant(project_id=3325fd0bf7e2439aa626201e2ae25c6b, group_id=300c8dafaf184bf59b61b6f89bdcf7de, role_id=abef78c185c74c16a2c7e8a170583f9f)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:00.006867 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e7c551-34e9-4769-892d-f771de2c927d None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:00.036197 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 210/411] 10.222.0.102 () {58 vars in 1447 bytes} [Thu Sep 1 17:17:59 2022] PUT /identity/v3/projects/3325fd0bf7e2439aa626201e2ae25c6b/groups/300c8dafaf184bf59b61b6f89bdcf7de/roles/abef78c185c74c16a2c7e8a170583f9f => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:18:00.036384 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86783da8-8bb4-4928-9abe-6c3058d7ed6b None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:00.080810 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83631f8c-1f02-48aa-86ed-aae0b7dae539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:00.081474 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83631f8c-1f02-48aa-86ed-aae0b7dae539 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:00.081754 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83631f8c-1f02-48aa-86ed-aae0b7dae539 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:00.082027 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83631f8c-1f02-48aa-86ed-aae0b7dae539 None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:00.082748 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83631f8c-1f02-48aa-86ed-aae0b7dae539 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:00.083047 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83631f8c-1f02-48aa-86ed-aae0b7dae539 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:00.084484 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83631f8c-1f02-48aa-86ed-aae0b7dae539 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:00.102323 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 202/412] 10.222.0.102 () {56 vars in 1104 bytes} [Thu Sep 1 17:17:59 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:18:00.108961 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e7c551-34e9-4769-892d-f771de2c927d None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:00.267827 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e7c551-34e9-4769-892d-f771de2c927d None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:00.290001 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:00.306549 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ea0d48-09cc-4998-944c-41ffda0b3df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:00.307191 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ea0d48-09cc-4998-944c-41ffda0b3df2 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:00.307469 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ea0d48-09cc-4998-944c-41ffda0b3df2 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:00.307743 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ea0d48-09cc-4998-944c-41ffda0b3df2 None admin] PATH_INFO: `/v3/groups/acbd924b0e244da4ae0f704418cd4e68` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:00.308861 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ea0d48-09cc-4998-944c-41ffda0b3df2 None admin] RBAC: Authorizing `identity:get_group(group_id=acbd924b0e244da4ae0f704418cd4e68)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:00.310320 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ea0d48-09cc-4998-944c-41ffda0b3df2 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:00.312184 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 211/413] 10.222.0.102 () {56 vars in 1185 bytes} [Thu Sep 1 17:18:00 2022] GET /identity/v3/groups/acbd924b0e244da4ae0f704418cd4e68 => generated 230 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:00.317774 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83631f8c-1f02-48aa-86ed-aae0b7dae539 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:00.447927 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83631f8c-1f02-48aa-86ed-aae0b7dae539 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:00.462619 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cf6eef-cc39-4493-8a4c-90f27f2ec0ff None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:00.463251 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cf6eef-cc39-4493-8a4c-90f27f2ec0ff None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:00.463734 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cf6eef-cc39-4493-8a4c-90f27f2ec0ff None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:00.466545 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 212/414] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:18:00 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Sep 01 17:18:00.470864 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b73a017-b351-4419-8143-1d89189095df None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:00.471376 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b73a017-b351-4419-8143-1d89189095df None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:00.471844 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b73a017-b351-4419-8143-1d89189095df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:00.493643 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-5b73a017-b351-4419-8143-1d89189095df None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:18:00.495182 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-5b73a017-b351-4419-8143-1d89189095df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:18:00.498004 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc66b01-0b11-4c44-98ca-6a985e1634ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:00.498652 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc66b01-0b11-4c44-98ca-6a985e1634ad None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:00.498935 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc66b01-0b11-4c44-98ca-6a985e1634ad None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:00.499210 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc66b01-0b11-4c44-98ca-6a985e1634ad None admin] PATH_INFO: `/v3/projects/ace9b3845813477eb192c158600d0beb` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:00.500389 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc66b01-0b11-4c44-98ca-6a985e1634ad None admin] RBAC: Authorizing `identity:get_project(project_id=ace9b3845813477eb192c158600d0beb)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:00.501974 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc66b01-0b11-4c44-98ca-6a985e1634ad None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:00.508660 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 203/415] 10.222.0.102 () {56 vars in 1191 bytes} [Thu Sep 1 17:18:00 2022] GET /identity/v3/projects/ace9b3845813477eb192c158600d0beb => generated 345 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:00.510837 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc66b01-0b11-4c44-98ca-6a985e1634ad None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:00.517799 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:00.628840 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 213/416] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:18:00 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 17:18:00.674662 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc66b01-0b11-4c44-98ca-6a985e1634ad None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:00.717084 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c93a346-955d-40dd-91e2-6cc866dd9135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:00.717789 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c93a346-955d-40dd-91e2-6cc866dd9135 None admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:00.718067 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c93a346-955d-40dd-91e2-6cc866dd9135 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:00.718343 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c93a346-955d-40dd-91e2-6cc866dd9135 None admin] PATH_INFO: `/v3/projects/ace9b3845813477eb192c158600d0beb/groups/acbd924b0e244da4ae0f704418cd4e68/roles/36577a6f46d642d4b81805ede9872490` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:00.720247 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c93a346-955d-40dd-91e2-6cc866dd9135 None admin] RBAC: Authorizing `identity:create_grant(project_id=ace9b3845813477eb192c158600d0beb, group_id=acbd924b0e244da4ae0f704418cd4e68, role_id=36577a6f46d642d4b81805ede9872490)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:00.721710 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c93a346-955d-40dd-91e2-6cc866dd9135 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:00.742067 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 204/417] 10.222.0.102 () {58 vars in 1447 bytes} [Thu Sep 1 17:18:00 2022] PUT /identity/v3/projects/ace9b3845813477eb192c158600d0beb/groups/acbd924b0e244da4ae0f704418cd4e68/roles/36577a6f46d642d4b81805ede9872490 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:18:00.785210 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3e71ec-bdbe-446c-ab6c-916746f0d941 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:00.785593 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3e71ec-bdbe-446c-ab6c-916746f0d941 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:00.785849 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3e71ec-bdbe-446c-ab6c-916746f0d941 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:00.787250 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 214/418] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:18:00 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Sep 01 17:18:00.792694 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe4e615-b9c4-428d-b997-d4587beb1eff None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:00.792981 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe4e615-b9c4-428d-b997-d4587beb1eff None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:00.793253 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe4e615-b9c4-428d-b997-d4587beb1eff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:00.818540 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-ffe4e615-b9c4-428d-b997-d4587beb1eff None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:18:00.819757 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-ffe4e615-b9c4-428d-b997-d4587beb1eff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:18:00.903797 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:00.924019 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 205/419] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:18:00 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:18:00.976084 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3e71ec-bdbe-446c-ab6c-916746f0d941 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:01.083957 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3e71ec-bdbe-446c-ab6c-916746f0d941 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:01.105449 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:01.120863 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb2461c-ec44-40f1-be0f-d03fff24f289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:01.121906 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb2461c-ec44-40f1-be0f-d03fff24f289 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:01.122402 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb2461c-ec44-40f1-be0f-d03fff24f289 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:01.122931 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb2461c-ec44-40f1-be0f-d03fff24f289 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:01.134683 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb2461c-ec44-40f1-be0f-d03fff24f289 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:18:01.139840 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-1eb2461c-ec44-40f1-be0f-d03fff24f289 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:18:01.139840 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:01.139840 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:01.139840 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:01.139840 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:01.139840 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:01.139840 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:01.139840 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:01.139840 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:01.139840 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:01.139840 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:01.139840 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:01.139840 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:18:01.139840 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:18:01.139840 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:18:01.139840 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:18:01.140717 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:18:01.140717 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:18:01.140717 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:18:01.140717 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:18:01.140717 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:01.140717 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:01.140717 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:01.140717 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:01.140717 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:01.140717 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:01.140717 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:01.140717 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:01.140717 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:01.140717 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:01.140717 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:01.140717 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:01.141538 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:01.141538 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:01.141538 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:01.141538 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:01.141538 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:18:01.141538 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:18:01.141538 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:18:01.141538 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:18:01.141538 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:18:01.141538 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:18:01.141538 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:18:01.141538 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:18:01.144138 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 215/420] 10.222.0.102 () {56 vars in 1104 bytes} [Thu Sep 1 17:18:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:01.150164 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe4e615-b9c4-428d-b997-d4587beb1eff None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:01.295168 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb2461c-ec44-40f1-be0f-d03fff24f289 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:01.335447 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe4e615-b9c4-428d-b997-d4587beb1eff None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:01.394623 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2ae14b-c029-443c-8e30-5d8fba079280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:01.395290 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2ae14b-c029-443c-8e30-5d8fba079280 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:01.395574 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2ae14b-c029-443c-8e30-5d8fba079280 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:01.395879 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2ae14b-c029-443c-8e30-5d8fba079280 None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:01.396510 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2ae14b-c029-443c-8e30-5d8fba079280 None admin] RBAC: Adding query filter params (name=member) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:01.396814 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2ae14b-c029-443c-8e30-5d8fba079280 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:01.398657 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2ae14b-c029-443c-8e30-5d8fba079280 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:01.407532 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb2461c-ec44-40f1-be0f-d03fff24f289 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:01.417217 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 206/421] 10.222.0.102 () {56 vars in 1106 bytes} [Thu Sep 1 17:18:01 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:01.425095 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2ae14b-c029-443c-8e30-5d8fba079280 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:01.450036 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e842b6-61c2-40cb-b27e-02573f4f942c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:01.451095 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e842b6-61c2-40cb-b27e-02573f4f942c None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:01.451650 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e842b6-61c2-40cb-b27e-02573f4f942c None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:01.452130 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e842b6-61c2-40cb-b27e-02573f4f942c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:01.463311 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e842b6-61c2-40cb-b27e-02573f4f942c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:18:01.468587 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-71e842b6-61c2-40cb-b27e-02573f4f942c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 17:18:01.468587 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:01.468587 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:01.468587 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:01.468587 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:01.468587 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:01.468587 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:01.468587 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:01.468587 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:01.468587 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:01.468587 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:01.468587 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:01.468587 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:18:01.468587 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:18:01.468587 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:18:01.468587 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:18:01.469478 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:18:01.469478 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:18:01.469478 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:18:01.469478 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:18:01.469478 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:01.469478 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:01.469478 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:01.469478 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:01.469478 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:01.469478 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:01.469478 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:01.469478 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:01.469478 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:01.469478 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:01.469478 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:01.469478 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:01.470243 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:01.470243 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:01.470243 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:01.470243 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:01.470243 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:18:01.470243 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:18:01.470243 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:18:01.470243 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:18:01.470243 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:18:01.470243 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:18:01.470243 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 17:18:01.470243 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:18:01.473572 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 216/422] 10.222.0.102 () {56 vars in 1119 bytes} [Thu Sep 1 17:18:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:18:01.486976 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e842b6-61c2-40cb-b27e-02573f4f942c None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:01.533708 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:01.585743 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:01.594600 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2ae14b-c029-443c-8e30-5d8fba079280 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:01.632868 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9d39e3-6a7e-47e7-bb8b-bf25cdaf23de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:01.633916 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9d39e3-6a7e-47e7-bb8b-bf25cdaf23de None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:01.634833 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9d39e3-6a7e-47e7-bb8b-bf25cdaf23de None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:01.635317 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9d39e3-6a7e-47e7-bb8b-bf25cdaf23de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:01.738579 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e842b6-61c2-40cb-b27e-02573f4f942c None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:01.758194 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9d39e3-6a7e-47e7-bb8b-bf25cdaf23de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:01.760584 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9d39e3-6a7e-47e7-bb8b-bf25cdaf23de None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:01.786087 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b414ab-eead-4f59-9ac4-a3e58ed47418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:01.786690 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b414ab-eead-4f59-9ac4-a3e58ed47418 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:01.786965 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b414ab-eead-4f59-9ac4-a3e58ed47418 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:01.787235 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b414ab-eead-4f59-9ac4-a3e58ed47418 None admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:01.787901 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b414ab-eead-4f59-9ac4-a3e58ed47418 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:01.788204 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b414ab-eead-4f59-9ac4-a3e58ed47418 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:01.789709 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b414ab-eead-4f59-9ac4-a3e58ed47418 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:01.798940 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 217/423] 10.222.0.102 () {56 vars in 1116 bytes} [Thu Sep 1 17:18:01 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:01.800520 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04dcfd4-e48f-4eaa-bca7-d777c9771f29 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:01.800733 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04dcfd4-e48f-4eaa-bca7-d777c9771f29 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:01.801019 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04dcfd4-e48f-4eaa-bca7-d777c9771f29 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:01.802874 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 218/424] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:18:01 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:18:01.809991 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04dcfd4-e48f-4eaa-bca7-d777c9771f29 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:01.839712 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:01.905825 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 207/425] 10.222.0.102 () {58 vars in 1287 bytes} [Thu Sep 1 17:18:01 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 01 17:18:01.905825 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aa5a27-dd9f-42ed-94cf-634cb6aba564 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:01.905825 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aa5a27-dd9f-42ed-94cf-634cb6aba564 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:01.905825 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aa5a27-dd9f-42ed-94cf-634cb6aba564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:01.922590 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-44aa5a27-dd9f-42ed-94cf-634cb6aba564 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:18:01.923848 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-44aa5a27-dd9f-42ed-94cf-634cb6aba564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:18:01.941399 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:01.947375 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04dcfd4-e48f-4eaa-bca7-d777c9771f29 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:01.993815 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f451417-2f97-4572-b0c9-88ffc2c5e9b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:01.995180 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f451417-2f97-4572-b0c9-88ffc2c5e9b2 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:01.995783 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f451417-2f97-4572-b0c9-88ffc2c5e9b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:01.996249 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f451417-2f97-4572-b0c9-88ffc2c5e9b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:02.002172 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 208/426] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:18:01 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:18:02.004315 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44aa5a27-dd9f-42ed-94cf-634cb6aba564 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:02.077216 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f451417-2f97-4572-b0c9-88ffc2c5e9b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:02.079240 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f451417-2f97-4572-b0c9-88ffc2c5e9b2 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:02.218417 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:02.224513 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 219/427] 10.222.0.102 () {58 vars in 1287 bytes} [Thu Sep 1 17:18:01 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 17:18:02.227424 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:02.230404 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f451417-2f97-4572-b0c9-88ffc2c5e9b2 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:02.275557 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44aa5a27-dd9f-42ed-94cf-634cb6aba564 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:02.345728 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f451417-2f97-4572-b0c9-88ffc2c5e9b2 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:02.372394 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef834d2-a2fc-4cb0-8b40-796588db74a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:02.373551 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef834d2-a2fc-4cb0-8b40-796588db74a9 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:02.374066 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef834d2-a2fc-4cb0-8b40-796588db74a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:02.374538 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef834d2-a2fc-4cb0-8b40-796588db74a9 None admin] PATH_INFO: `/v3/projects/8e88c67b942f45108c95a833547f9bc8` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:02.376083 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef834d2-a2fc-4cb0-8b40-796588db74a9 None admin] RBAC: Authorizing `identity:get_project(project_id=8e88c67b942f45108c95a833547f9bc8)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:02.377921 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef834d2-a2fc-4cb0-8b40-796588db74a9 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:02.384801 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 209/428] 10.222.0.102 () {56 vars in 1191 bytes} [Thu Sep 1 17:18:02 2022] GET /identity/v3/projects/8e88c67b942f45108c95a833547f9bc8 => generated 299 bytes in 377 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:02.384801 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef834d2-a2fc-4cb0-8b40-796588db74a9 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:02.388493 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d58ec2-c5b1-4035-98af-58057e1edf28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:02.390018 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d58ec2-c5b1-4035-98af-58057e1edf28 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:02.390306 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d58ec2-c5b1-4035-98af-58057e1edf28 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:02.390582 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d58ec2-c5b1-4035-98af-58057e1edf28 None admin] PATH_INFO: `/v3/projects/8e88c67b942f45108c95a833547f9bc8` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:02.391805 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d58ec2-c5b1-4035-98af-58057e1edf28 None admin] RBAC: Authorizing `identity:get_project(project_id=8e88c67b942f45108c95a833547f9bc8)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:02.393259 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d58ec2-c5b1-4035-98af-58057e1edf28 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:02.395308 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 220/429] 10.222.0.102 () {56 vars in 1191 bytes} [Thu Sep 1 17:18:02 2022] GET /identity/v3/projects/8e88c67b942f45108c95a833547f9bc8 => generated 299 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:02.396655 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d58ec2-c5b1-4035-98af-58057e1edf28 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:02.477561 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:02.494088 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef834d2-a2fc-4cb0-8b40-796588db74a9 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:02.514921 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d58ec2-c5b1-4035-98af-58057e1edf28 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:02.528873 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a804cb33-bb8f-4255-a612-a2800b692766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:02.529536 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a804cb33-bb8f-4255-a612-a2800b692766 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:02.529821 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a804cb33-bb8f-4255-a612-a2800b692766 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:02.530091 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a804cb33-bb8f-4255-a612-a2800b692766 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:02.540760 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a804cb33-bb8f-4255-a612-a2800b692766 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:18:02.545660 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-a804cb33-bb8f-4255-a612-a2800b692766 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:18:02.545660 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:02.545660 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:02.545660 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:02.545660 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:02.545660 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:02.545660 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:02.545660 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:02.545660 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:02.545660 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:02.545660 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:02.545660 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:02.545660 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:18:02.545660 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:18:02.545660 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:18:02.545660 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:18:02.546629 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:18:02.546629 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:18:02.546629 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:18:02.546629 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:18:02.546629 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:02.546629 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:02.546629 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:02.546629 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:02.546629 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:02.546629 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:02.546629 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:02.546629 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:02.546629 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:02.546629 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:02.546629 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:02.546629 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:02.547407 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:02.547407 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:02.547407 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:02.547407 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:02.547407 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:18:02.547407 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:18:02.547407 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:18:02.547407 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:18:02.547407 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:18:02.547407 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:18:02.547407 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:18:02.547407 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:18:02.547407 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 210/430] 10.222.0.102 () {56 vars in 1104 bytes} [Thu Sep 1 17:18:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:02.552718 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a804cb33-bb8f-4255-a612-a2800b692766 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:02.556971 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3224a075-3a69-4ca0-920d-3535172605f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:02.557895 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3224a075-3a69-4ca0-920d-3535172605f3 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:02.558218 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3224a075-3a69-4ca0-920d-3535172605f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:02.558486 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3224a075-3a69-4ca0-920d-3535172605f3 None admin] PATH_INFO: `/v3/groups/300c8dafaf184bf59b61b6f89bdcf7de` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:02.559719 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3224a075-3a69-4ca0-920d-3535172605f3 None admin] RBAC: Authorizing `identity:get_group(group_id=300c8dafaf184bf59b61b6f89bdcf7de)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:02.561279 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3224a075-3a69-4ca0-920d-3535172605f3 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:02.567610 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 221/431] 10.222.0.102 () {56 vars in 1185 bytes} [Thu Sep 1 17:18:02 2022] GET /identity/v3/groups/300c8dafaf184bf59b61b6f89bdcf7de => generated 227 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:02.567610 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9475e7cc-ad20-49af-8487-92f0e0bfddbb None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:02.567610 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9475e7cc-ad20-49af-8487-92f0e0bfddbb None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:02.567610 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9475e7cc-ad20-49af-8487-92f0e0bfddbb None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:02.567610 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 222/432] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:18:02 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:18:02.567610 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9475e7cc-ad20-49af-8487-92f0e0bfddbb None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:02.575394 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:02.668578 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a804cb33-bb8f-4255-a612-a2800b692766 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:02.675851 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9475e7cc-ad20-49af-8487-92f0e0bfddbb None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:02.702997 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b91e6e-8891-4b56-986f-0455413efcd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:02.703629 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b91e6e-8891-4b56-986f-0455413efcd8 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:02.703910 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b91e6e-8891-4b56-986f-0455413efcd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:02.704182 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b91e6e-8891-4b56-986f-0455413efcd8 None admin] PATH_INFO: `/v3/groups/300c8dafaf184bf59b61b6f89bdcf7de` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:02.705290 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b91e6e-8891-4b56-986f-0455413efcd8 None admin] RBAC: Authorizing `identity:get_group(group_id=300c8dafaf184bf59b61b6f89bdcf7de)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:02.706756 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b91e6e-8891-4b56-986f-0455413efcd8 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:02.713532 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 211/433] 10.222.0.102 () {56 vars in 1185 bytes} [Thu Sep 1 17:18:02 2022] GET /identity/v3/groups/300c8dafaf184bf59b61b6f89bdcf7de => generated 227 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:02.713532 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b91e6e-8891-4b56-986f-0455413efcd8 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:02.715960 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4e6eca-191d-4d9e-9048-d04c9e283b3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:02.716680 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4e6eca-191d-4d9e-9048-d04c9e283b3a None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:02.716960 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4e6eca-191d-4d9e-9048-d04c9e283b3a None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:02.717231 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4e6eca-191d-4d9e-9048-d04c9e283b3a None admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:02.717928 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4e6eca-191d-4d9e-9048-d04c9e283b3a None admin] RBAC: Adding query filter params (name=member) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:02.718231 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4e6eca-191d-4d9e-9048-d04c9e283b3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:02.719658 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4e6eca-191d-4d9e-9048-d04c9e283b3a None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:02.721985 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:02.735227 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 223/434] 10.222.0.102 () {56 vars in 1106 bytes} [Thu Sep 1 17:18:02 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:02.735227 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c212d56d-ec87-4359-8900-8e714b2e67eb None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:02.735227 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c212d56d-ec87-4359-8900-8e714b2e67eb None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:02.735227 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c212d56d-ec87-4359-8900-8e714b2e67eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:02.755576 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-c212d56d-ec87-4359-8900-8e714b2e67eb None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:18:02.757567 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-c212d56d-ec87-4359-8900-8e714b2e67eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:18:02.804745 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:02.826829 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b91e6e-8891-4b56-986f-0455413efcd8 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:02.831921 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 224/435] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:18:02 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:18:02.834298 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c212d56d-ec87-4359-8900-8e714b2e67eb None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:02.886901 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdd12f8-c132-4e23-af59-67ba2e014962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:02.893167 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdd12f8-c132-4e23-af59-67ba2e014962 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:02.893481 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdd12f8-c132-4e23-af59-67ba2e014962 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:02.893755 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdd12f8-c132-4e23-af59-67ba2e014962 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:02.894496 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdd12f8-c132-4e23-af59-67ba2e014962 None admin] RBAC: Adding query filter params (group.id=300c8dafaf184bf59b61b6f89bdcf7de, role.id=b75cb360925e4ba5a65f14965c103ffd, scope.project.id=8e88c67b942f45108c95a833547f9bc8) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:02.894840 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdd12f8-c132-4e23-af59-67ba2e014962 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:02.896311 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdd12f8-c132-4e23-af59-67ba2e014962 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:02.909896 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 212/436] 10.222.0.102 () {56 vars in 1381 bytes} [Thu Sep 1 17:18:02 2022] GET /identity/v3/role_assignments?group.id=300c8dafaf184bf59b61b6f89bdcf7de&scope.project.id=8e88c67b942f45108c95a833547f9bc8&role.id=b75cb360925e4ba5a65f14965c103ffd => generated 613 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:02.910919 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdd12f8-c132-4e23-af59-67ba2e014962 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:02.956858 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c212d56d-ec87-4359-8900-8e714b2e67eb None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:02.999203 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f0b120-a0af-449f-a6c8-37fbf1c27683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:03.000223 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f0b120-a0af-449f-a6c8-37fbf1c27683 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:03.000755 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f0b120-a0af-449f-a6c8-37fbf1c27683 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:03.001236 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f0b120-a0af-449f-a6c8-37fbf1c27683 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:03.002331 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f0b120-a0af-449f-a6c8-37fbf1c27683 None admin] RBAC: Adding query filter params (group.id=300c8dafaf184bf59b61b6f89bdcf7de, role.id=abef78c185c74c16a2c7e8a170583f9f, scope.project.id=8e88c67b942f45108c95a833547f9bc8) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:03.002858 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f0b120-a0af-449f-a6c8-37fbf1c27683 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:03.004735 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f0b120-a0af-449f-a6c8-37fbf1c27683 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:03.009546 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:03.012339 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 225/437] 10.222.0.102 () {56 vars in 1381 bytes} [Thu Sep 1 17:18:02 2022] GET /identity/v3/role_assignments?group.id=300c8dafaf184bf59b61b6f89bdcf7de&scope.project.id=8e88c67b942f45108c95a833547f9bc8&role.id=abef78c185c74c16a2c7e8a170583f9f => generated 613 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:03.018024 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6312b137-bf4e-4fe4-975a-440d06ee50f5 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:03.018832 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6312b137-bf4e-4fe4-975a-440d06ee50f5 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:03.019455 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6312b137-bf4e-4fe4-975a-440d06ee50f5 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:03.021885 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 226/438] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:18:03 2022] GET /identity => generated 271 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:18:03.029525 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6de638-e716-4a17-897a-985ea8f19072 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:03.029525 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6de638-e716-4a17-897a-985ea8f19072 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:03.029525 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6de638-e716-4a17-897a-985ea8f19072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:03.068374 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-fb6de638-e716-4a17-897a-985ea8f19072 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:18:03.069649 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-fb6de638-e716-4a17-897a-985ea8f19072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:18:03.134636 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdd12f8-c132-4e23-af59-67ba2e014962 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:03.178476 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd68783-2fc4-4ebc-bf40-6acc818a494c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:03.179388 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd68783-2fc4-4ebc-bf40-6acc818a494c None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:03.179988 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd68783-2fc4-4ebc-bf40-6acc818a494c None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:03.180265 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd68783-2fc4-4ebc-bf40-6acc818a494c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:03.188708 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 227/439] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:18:03 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 17:18:03.189849 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6de638-e716-4a17-897a-985ea8f19072 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:03.200643 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:03.282547 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd68783-2fc4-4ebc-bf40-6acc818a494c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:03.284325 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd68783-2fc4-4ebc-bf40-6acc818a494c None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:03.316032 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6de638-e716-4a17-897a-985ea8f19072 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:03.346706 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:03.360369 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0714de-4526-4ea3-a131-f3c2fc578ffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:03.361896 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0714de-4526-4ea3-a131-f3c2fc578ffe None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:03.362179 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0714de-4526-4ea3-a131-f3c2fc578ffe None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:03.362454 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0714de-4526-4ea3-a131-f3c2fc578ffe None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:03.381998 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0714de-4526-4ea3-a131-f3c2fc578ffe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:18:03.390141 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-fe0714de-4526-4ea3-a131-f3c2fc578ffe None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 17:18:03.390141 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:03.390141 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:03.390141 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:03.390141 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:03.390141 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:03.390141 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:03.390141 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:03.390141 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:03.390141 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:03.390141 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:03.390141 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:03.390141 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:18:03.390141 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:18:03.390141 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:18:03.390141 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:18:03.391037 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:18:03.391037 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:18:03.391037 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:18:03.391037 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:18:03.391037 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:03.391037 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:03.391037 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:03.391037 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:03.391037 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:03.391037 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:03.391037 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:03.391037 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:03.391037 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:03.391037 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:03.391037 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:03.391037 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:03.391821 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:03.391821 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:03.391821 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:03.391821 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:03.391821 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:18:03.391821 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:18:03.391821 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:18:03.391821 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:18:03.391821 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:18:03.391821 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:18:03.391821 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 17:18:03.391821 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:18:03.391821 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 228/440] 10.222.0.102 () {56 vars in 1119 bytes} [Thu Sep 1 17:18:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:03.397692 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0714de-4526-4ea3-a131-f3c2fc578ffe None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:03.423229 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 213/441] 10.222.0.102 () {58 vars in 1287 bytes} [Thu Sep 1 17:18:02 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 513 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 17:18:03.430855 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd68783-2fc4-4ebc-bf40-6acc818a494c None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:03.490643 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:03.525646 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0714de-4526-4ea3-a131-f3c2fc578ffe None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:03.553985 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd68783-2fc4-4ebc-bf40-6acc818a494c None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:03.564754 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129d382f-1811-4b36-826c-a180b5c6da5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:03.565792 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129d382f-1811-4b36-826c-a180b5c6da5c None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:03.566071 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129d382f-1811-4b36-826c-a180b5c6da5c None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:03.566343 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129d382f-1811-4b36-826c-a180b5c6da5c None admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:03.567969 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129d382f-1811-4b36-826c-a180b5c6da5c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:03.568291 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129d382f-1811-4b36-826c-a180b5c6da5c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:03.571941 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129d382f-1811-4b36-826c-a180b5c6da5c None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:03.579144 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 229/442] 10.222.0.102 () {56 vars in 1116 bytes} [Thu Sep 1 17:18:03 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:18:03.580050 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129d382f-1811-4b36-826c-a180b5c6da5c None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:03.596995 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af33a5d-369a-44e7-91c2-e2c9083131de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:03.597748 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af33a5d-369a-44e7-91c2-e2c9083131de None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:03.598099 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af33a5d-369a-44e7-91c2-e2c9083131de None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:03.598333 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af33a5d-369a-44e7-91c2-e2c9083131de None admin] PATH_INFO: `/v3/projects/3325fd0bf7e2439aa626201e2ae25c6b` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:03.599436 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af33a5d-369a-44e7-91c2-e2c9083131de None admin] RBAC: Authorizing `identity:get_project(project_id=3325fd0bf7e2439aa626201e2ae25c6b)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:03.600901 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af33a5d-369a-44e7-91c2-e2c9083131de None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:03.602612 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 214/443] 10.222.0.102 () {56 vars in 1191 bytes} [Thu Sep 1 17:18:03 2022] GET /identity/v3/projects/3325fd0bf7e2439aa626201e2ae25c6b => generated 303 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:03.603661 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af33a5d-369a-44e7-91c2-e2c9083131de None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:03.633737 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:03.694661 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129d382f-1811-4b36-826c-a180b5c6da5c None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:03.708657 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af33a5d-369a-44e7-91c2-e2c9083131de None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:03.721758 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:03.728095 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b80322-7795-40a0-8734-9da520c16628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:03.728711 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b80322-7795-40a0-8734-9da520c16628 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:03.728991 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b80322-7795-40a0-8734-9da520c16628 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:03.729263 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b80322-7795-40a0-8734-9da520c16628 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:03.739727 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b80322-7795-40a0-8734-9da520c16628 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:18:03.743907 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fa221a-0246-4609-af29-0d32dd3abb1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:03.744509 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fa221a-0246-4609-af29-0d32dd3abb1f None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:03.744859 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fa221a-0246-4609-af29-0d32dd3abb1f None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:03.744973 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-85b80322-7795-40a0-8734-9da520c16628 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:18:03.744973 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:03.744973 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:03.744973 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:03.744973 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:03.744973 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:03.744973 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:03.744973 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:03.744973 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:03.744973 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:03.744973 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:03.744973 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:03.744973 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:18:03.744973 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:18:03.744973 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:18:03.744973 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:18:03.745984 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:18:03.745984 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:18:03.745984 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:18:03.745984 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:18:03.745984 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:03.745984 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:03.745984 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:03.745984 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:03.745984 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:03.745984 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:03.745984 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:03.745984 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:03.745984 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:03.745984 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:03.745984 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:03.745984 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:03.746948 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:03.746948 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:03.746948 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:03.746948 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:03.746948 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:18:03.746948 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:18:03.746948 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:18:03.746948 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:18:03.746948 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:18:03.746948 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:18:03.746948 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:18:03.746948 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:18:03.747660 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fa221a-0246-4609-af29-0d32dd3abb1f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:03.750750 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 230/444] 10.222.0.102 () {56 vars in 1101 bytes} [Thu Sep 1 17:18:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:03.751572 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b80322-7795-40a0-8734-9da520c16628 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:03.817995 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fa221a-0246-4609-af29-0d32dd3abb1f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:03.819447 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fa221a-0246-4609-af29-0d32dd3abb1f None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:03.848542 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b80322-7795-40a0-8734-9da520c16628 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:03.852595 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:03.885271 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbdc616-fcf0-472f-adeb-e55b74fceab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:03.885920 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbdc616-fcf0-472f-adeb-e55b74fceab3 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:03.886195 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbdc616-fcf0-472f-adeb-e55b74fceab3 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:03.886466 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbdc616-fcf0-472f-adeb-e55b74fceab3 None admin] PATH_INFO: `/v3/groups/300c8dafaf184bf59b61b6f89bdcf7de` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:03.887564 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbdc616-fcf0-472f-adeb-e55b74fceab3 None admin] RBAC: Authorizing `identity:get_group(group_id=300c8dafaf184bf59b61b6f89bdcf7de)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:03.888977 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbdc616-fcf0-472f-adeb-e55b74fceab3 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:03.890585 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 231/445] 10.222.0.102 () {56 vars in 1185 bytes} [Thu Sep 1 17:18:03 2022] GET /identity/v3/groups/300c8dafaf184bf59b61b6f89bdcf7de => generated 227 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:03.891335 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbdc616-fcf0-472f-adeb-e55b74fceab3 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:03.918061 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 215/446] 10.222.0.102 () {58 vars in 1287 bytes} [Thu Sep 1 17:18:03 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 17:18:03.918902 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fa221a-0246-4609-af29-0d32dd3abb1f None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:03.933241 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:03.990784 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbdc616-fcf0-472f-adeb-e55b74fceab3 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:04.022668 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fa221a-0246-4609-af29-0d32dd3abb1f None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:04.024581 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acbc33b-71b7-48e7-bed9-5488ddb98460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:04.025475 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acbc33b-71b7-48e7-bed9-5488ddb98460 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:04.025589 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acbc33b-71b7-48e7-bed9-5488ddb98460 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:04.025816 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acbc33b-71b7-48e7-bed9-5488ddb98460 None admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:04.026470 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acbc33b-71b7-48e7-bed9-5488ddb98460 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:04.026771 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acbc33b-71b7-48e7-bed9-5488ddb98460 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:04.028125 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acbc33b-71b7-48e7-bed9-5488ddb98460 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:04.036142 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 232/447] 10.222.0.102 () {56 vars in 1104 bytes} [Thu Sep 1 17:18:03 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:04.036931 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acbc33b-71b7-48e7-bed9-5488ddb98460 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:04.056566 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c02047-7e4e-401e-b5c4-724cda90b862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:04.057170 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c02047-7e4e-401e-b5c4-724cda90b862 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:04.057502 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c02047-7e4e-401e-b5c4-724cda90b862 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:04.057763 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c02047-7e4e-401e-b5c4-724cda90b862 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:04.058487 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c02047-7e4e-401e-b5c4-724cda90b862 None admin] RBAC: Adding query filter params (group.id=300c8dafaf184bf59b61b6f89bdcf7de, role.id=b75cb360925e4ba5a65f14965c103ffd, scope.project.id=3325fd0bf7e2439aa626201e2ae25c6b) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:04.058852 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c02047-7e4e-401e-b5c4-724cda90b862 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:04.060362 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c02047-7e4e-401e-b5c4-724cda90b862 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:04.066580 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 216/448] 10.222.0.102 () {56 vars in 1381 bytes} [Thu Sep 1 17:18:03 2022] GET /identity/v3/role_assignments?group.id=300c8dafaf184bf59b61b6f89bdcf7de&scope.project.id=3325fd0bf7e2439aa626201e2ae25c6b&role.id=b75cb360925e4ba5a65f14965c103ffd => generated 613 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:04.067840 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c02047-7e4e-401e-b5c4-724cda90b862 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:04.072756 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:04.152522 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acbc33b-71b7-48e7-bed9-5488ddb98460 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:04.157297 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:04.183350 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c02047-7e4e-401e-b5c4-724cda90b862 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:04.195329 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f5af96-941c-4985-980a-221555a02c00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:04.195975 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f5af96-941c-4985-980a-221555a02c00 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:04.196261 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f5af96-941c-4985-980a-221555a02c00 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:04.196538 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f5af96-941c-4985-980a-221555a02c00 None admin] PATH_INFO: `/v3/projects/3325fd0bf7e2439aa626201e2ae25c6b` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:04.197894 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f5af96-941c-4985-980a-221555a02c00 None admin] RBAC: Authorizing `identity:get_project(project_id=3325fd0bf7e2439aa626201e2ae25c6b)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:04.199512 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f5af96-941c-4985-980a-221555a02c00 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:04.205569 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 233/449] 10.222.0.102 () {56 vars in 1191 bytes} [Thu Sep 1 17:18:04 2022] GET /identity/v3/projects/3325fd0bf7e2439aa626201e2ae25c6b => generated 303 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:04.206599 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f5af96-941c-4985-980a-221555a02c00 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:04.229803 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a35781-d39f-49da-84b0-07556d714144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:04.230736 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a35781-d39f-49da-84b0-07556d714144 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:04.231961 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a35781-d39f-49da-84b0-07556d714144 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:04.231961 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a35781-d39f-49da-84b0-07556d714144 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:04.318277 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a35781-d39f-49da-84b0-07556d714144 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:04.319670 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a35781-d39f-49da-84b0-07556d714144 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:04.334129 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f5af96-941c-4985-980a-221555a02c00 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:04.335675 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:04.377926 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7df7484-459e-4b8b-af78-ef5c0c785450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:04.378680 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7df7484-459e-4b8b-af78-ef5c0c785450 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:04.378962 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7df7484-459e-4b8b-af78-ef5c0c785450 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:04.379274 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7df7484-459e-4b8b-af78-ef5c0c785450 None admin] PATH_INFO: `/v3/groups/300c8dafaf184bf59b61b6f89bdcf7de` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:04.381941 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7df7484-459e-4b8b-af78-ef5c0c785450 None admin] RBAC: Authorizing `identity:get_group(group_id=300c8dafaf184bf59b61b6f89bdcf7de)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:04.382249 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7df7484-459e-4b8b-af78-ef5c0c785450 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:04.385344 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 234/450] 10.222.0.102 () {56 vars in 1185 bytes} [Thu Sep 1 17:18:04 2022] GET /identity/v3/groups/300c8dafaf184bf59b61b6f89bdcf7de => generated 227 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:18:04.389659 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7df7484-459e-4b8b-af78-ef5c0c785450 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:04.420322 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:04.447340 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 217/451] 10.222.0.102 () {58 vars in 1287 bytes} [Thu Sep 1 17:18:04 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 17:18:04.453246 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a35781-d39f-49da-84b0-07556d714144 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:04.497587 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7df7484-459e-4b8b-af78-ef5c0c785450 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:04.531012 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e30841-788f-42ee-9257-b658632bf609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:04.531611 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e30841-788f-42ee-9257-b658632bf609 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:04.531890 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e30841-788f-42ee-9257-b658632bf609 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:04.532162 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e30841-788f-42ee-9257-b658632bf609 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:04.532876 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e30841-788f-42ee-9257-b658632bf609 None admin] RBAC: Adding query filter params (group.id=300c8dafaf184bf59b61b6f89bdcf7de, role.id=abef78c185c74c16a2c7e8a170583f9f, scope.project.id=3325fd0bf7e2439aa626201e2ae25c6b) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:04.533183 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e30841-788f-42ee-9257-b658632bf609 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:04.534657 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e30841-788f-42ee-9257-b658632bf609 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:04.540239 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 235/452] 10.222.0.102 () {56 vars in 1381 bytes} [Thu Sep 1 17:18:04 2022] GET /identity/v3/role_assignments?group.id=300c8dafaf184bf59b61b6f89bdcf7de&scope.project.id=3325fd0bf7e2439aa626201e2ae25c6b&role.id=abef78c185c74c16a2c7e8a170583f9f => generated 613 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:18:04.557730 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a35781-d39f-49da-84b0-07556d714144 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:04.595857 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79334b6-55fe-485b-bffe-e5b4b3f12b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:04.596424 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79334b6-55fe-485b-bffe-e5b4b3f12b55 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:04.596707 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79334b6-55fe-485b-bffe-e5b4b3f12b55 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:04.596982 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79334b6-55fe-485b-bffe-e5b4b3f12b55 None admin] PATH_INFO: `/v3/projects/ace9b3845813477eb192c158600d0beb` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:04.598797 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79334b6-55fe-485b-bffe-e5b4b3f12b55 None admin] RBAC: Authorizing `identity:get_project(project_id=ace9b3845813477eb192c158600d0beb)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:04.600269 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79334b6-55fe-485b-bffe-e5b4b3f12b55 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:04.602012 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 218/453] 10.222.0.102 () {56 vars in 1191 bytes} [Thu Sep 1 17:18:04 2022] GET /identity/v3/projects/ace9b3845813477eb192c158600d0beb => generated 345 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:18:04.608113 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e30841-788f-42ee-9257-b658632bf609 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:04.644048 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:04.711650 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e30841-788f-42ee-9257-b658632bf609 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:04.745467 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3026ab8f-d210-4ebe-a9e7-3236943623c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:04.746092 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3026ab8f-d210-4ebe-a9e7-3236943623c7 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:04.746369 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3026ab8f-d210-4ebe-a9e7-3236943623c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:04.746639 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3026ab8f-d210-4ebe-a9e7-3236943623c7 None admin] PATH_INFO: `/v3/groups/acbd924b0e244da4ae0f704418cd4e68` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:04.747726 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3026ab8f-d210-4ebe-a9e7-3236943623c7 None admin] RBAC: Authorizing `identity:get_group(group_id=acbd924b0e244da4ae0f704418cd4e68)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:04.749210 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3026ab8f-d210-4ebe-a9e7-3236943623c7 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:04.750725 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 236/454] 10.222.0.102 () {56 vars in 1185 bytes} [Thu Sep 1 17:18:04 2022] GET /identity/v3/groups/acbd924b0e244da4ae0f704418cd4e68 => generated 230 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:04.756652 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79334b6-55fe-485b-bffe-e5b4b3f12b55 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:04.803908 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:04.867314 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79334b6-55fe-485b-bffe-e5b4b3f12b55 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:04.902519 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04e128a-6364-4510-9a49-205fb2b9a136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:04.903512 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04e128a-6364-4510-9a49-205fb2b9a136 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:04.903909 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04e128a-6364-4510-9a49-205fb2b9a136 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:04.904185 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04e128a-6364-4510-9a49-205fb2b9a136 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:04.904906 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04e128a-6364-4510-9a49-205fb2b9a136 None admin] RBAC: Adding query filter params (group.id=acbd924b0e244da4ae0f704418cd4e68, role.id=36577a6f46d642d4b81805ede9872490, scope.project.id=ace9b3845813477eb192c158600d0beb) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:04.905208 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04e128a-6364-4510-9a49-205fb2b9a136 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:04.906701 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04e128a-6364-4510-9a49-205fb2b9a136 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:04.915274 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 219/455] 10.222.0.102 () {56 vars in 1381 bytes} [Thu Sep 1 17:18:04 2022] GET /identity/v3/role_assignments?group.id=acbd924b0e244da4ae0f704418cd4e68&scope.project.id=ace9b3845813477eb192c158600d0beb&role.id=36577a6f46d642d4b81805ede9872490 => generated 613 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:07.194063 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28f7fd3-6af2-4163-8197-3fa8d049fe36 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:07.194541 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28f7fd3-6af2-4163-8197-3fa8d049fe36 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:07.194541 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28f7fd3-6af2-4163-8197-3fa8d049fe36 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:07.196122 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 237/456] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:18:07 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:18:07.201343 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2d899b-27f8-402f-a4f5-4e124b6ac29b None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:07.202152 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2d899b-27f8-402f-a4f5-4e124b6ac29b None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:07.202644 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2d899b-27f8-402f-a4f5-4e124b6ac29b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:07.230372 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-3f2d899b-27f8-402f-a4f5-4e124b6ac29b None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:18:07.232088 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-3f2d899b-27f8-402f-a4f5-4e124b6ac29b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:18:07.322568 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 220/457] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:18:07 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:18:07.523689 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfd3eb7-9b80-4422-bd16-bb173619c855 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:07.524356 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfd3eb7-9b80-4422-bd16-bb173619c855 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:07.524854 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfd3eb7-9b80-4422-bd16-bb173619c855 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:07.526670 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 238/458] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:18:07 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:18:07.532235 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686a9a28-30d7-4e57-a6ab-74def7a83533 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:07.532742 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686a9a28-30d7-4e57-a6ab-74def7a83533 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:07.533232 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686a9a28-30d7-4e57-a6ab-74def7a83533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:07.558280 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-686a9a28-30d7-4e57-a6ab-74def7a83533 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:18:07.559792 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-686a9a28-30d7-4e57-a6ab-74def7a83533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:18:07.600658 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:07.658107 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 221/459] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:18:07 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:18:07.766596 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686a9a28-30d7-4e57-a6ab-74def7a83533 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:07.766814 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f14cb67-be68-4939-97fc-e0435e6fb864 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:07.766814 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f14cb67-be68-4939-97fc-e0435e6fb864 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:07.766950 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f14cb67-be68-4939-97fc-e0435e6fb864 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:07.769374 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 239/460] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:18:07 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Sep 01 17:18:07.807115 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03af34c-2239-4cb2-81d6-ddca0dc673ec None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:07.807739 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03af34c-2239-4cb2-81d6-ddca0dc673ec None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:07.808211 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03af34c-2239-4cb2-81d6-ddca0dc673ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:07.842356 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-e03af34c-2239-4cb2-81d6-ddca0dc673ec None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:18:07.843934 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-e03af34c-2239-4cb2-81d6-ddca0dc673ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:18:07.886694 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686a9a28-30d7-4e57-a6ab-74def7a83533 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:07.923973 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c7c56a-511d-4b90-97a1-adc958529049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:07.924607 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c7c56a-511d-4b90-97a1-adc958529049 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:07.924882 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c7c56a-511d-4b90-97a1-adc958529049 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:07.925154 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c7c56a-511d-4b90-97a1-adc958529049 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:07.936568 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c7c56a-511d-4b90-97a1-adc958529049 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:07.938129 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c7c56a-511d-4b90-97a1-adc958529049 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:07.950571 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 240/461] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:18:07 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:18:07.952208 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd9f905-fa90-4059-bee8-ca88d0e3d3e5 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:07.952711 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd9f905-fa90-4059-bee8-ca88d0e3d3e5 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:07.953211 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd9f905-fa90-4059-bee8-ca88d0e3d3e5 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:07.954937 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 241/462] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:18:07 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:18:07.955041 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-94c7c56a-511d-4b90-97a1-adc958529049 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:07.955041 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:07.955041 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:07.955041 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:07.955041 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:07.955041 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:07.955041 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:07.955041 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:07.955041 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:07.955041 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:07.955041 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:07.955041 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:07.955041 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:18:07.955041 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:18:07.955041 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:18:07.955041 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:18:07.955873 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:07.955873 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:07.955873 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:07.955873 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:07.955873 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:07.955873 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:07.955873 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:07.955873 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:07.955873 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:07.955873 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:07.955873 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:07.955873 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:07.955873 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:07.955873 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:07.955873 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:07.955873 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:07.955873 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:18:07.956692 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:18:07.956692 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:07.956692 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:18:07.960214 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 222/463] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:18:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:07.961550 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d5da57-b227-42b7-8d7c-bb3fb01791a2 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:07.962056 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d5da57-b227-42b7-8d7c-bb3fb01791a2 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:07.962534 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d5da57-b227-42b7-8d7c-bb3fb01791a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:07.973336 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c7c56a-511d-4b90-97a1-adc958529049 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:07.996969 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-e8d5da57-b227-42b7-8d7c-bb3fb01791a2 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:18:08.000654 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-e8d5da57-b227-42b7-8d7c-bb3fb01791a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:18:08.003688 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:08.043129 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:08.092296 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c7c56a-511d-4b90-97a1-adc958529049 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:08.098950 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 242/464] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:18:07 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:18:08.100401 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d5da57-b227-42b7-8d7c-bb3fb01791a2 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:08.186009 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b89b48-ab84-4ab6-9d99-51000338d965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:08.186692 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b89b48-ab84-4ab6-9d99-51000338d965 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:08.186981 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b89b48-ab84-4ab6-9d99-51000338d965 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:08.187257 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b89b48-ab84-4ab6-9d99-51000338d965 None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:08.187900 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b89b48-ab84-4ab6-9d99-51000338d965 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:08.188206 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b89b48-ab84-4ab6-9d99-51000338d965 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:08.189840 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b89b48-ab84-4ab6-9d99-51000338d965 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:08.213543 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d5da57-b227-42b7-8d7c-bb3fb01791a2 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:08.216580 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 223/465] 10.222.0.102 () {56 vars in 1114 bytes} [Thu Sep 1 17:18:07 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:18:08.221873 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b89b48-ab84-4ab6-9d99-51000338d965 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:08.258905 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e785e2-82e3-4f33-8b67-4d80de878d32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:08.259826 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e785e2-82e3-4f33-8b67-4d80de878d32 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:08.260113 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e785e2-82e3-4f33-8b67-4d80de878d32 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:08.260384 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e785e2-82e3-4f33-8b67-4d80de878d32 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:08.274740 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e785e2-82e3-4f33-8b67-4d80de878d32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:08.276674 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e785e2-82e3-4f33-8b67-4d80de878d32 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:08.298688 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-82e785e2-82e3-4f33-8b67-4d80de878d32 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:08.298688 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:08.298688 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:08.298688 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:08.298688 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:08.298688 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:08.298688 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:08.298688 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:08.298688 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:08.298688 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:08.298688 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:08.298688 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:08.298688 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:18:08.298688 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:18:08.298688 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:18:08.298688 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:18:08.299612 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:08.299612 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:08.299612 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:08.299612 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:08.299612 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:08.299612 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:08.299612 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:08.299612 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:08.299612 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:08.299612 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:08.299612 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:08.299612 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:08.299612 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:08.299612 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:08.299612 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:08.299612 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:08.299612 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:18:08.300428 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:18:08.300428 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:08.300428 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:18:08.300596 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 243/466] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:18:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:08.306089 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e785e2-82e3-4f33-8b67-4d80de878d32 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:08.324833 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:08.332012 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:08.366353 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b89b48-ab84-4ab6-9d99-51000338d965 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:08.402380 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67ac2ff-6142-4310-a092-d762defd759f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:08.403153 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67ac2ff-6142-4310-a092-d762defd759f None admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:08.403499 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67ac2ff-6142-4310-a092-d762defd759f None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:08.403836 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67ac2ff-6142-4310-a092-d762defd759f None admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:08.404547 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67ac2ff-6142-4310-a092-d762defd759f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:08.406063 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67ac2ff-6142-4310-a092-d762defd759f None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:08.431711 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 224/467] 10.222.0.102 () {60 vars in 1137 bytes} [Thu Sep 1 17:18:08 2022] POST /identity/v3/users => generated 247 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:18:08.432744 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67ac2ff-6142-4310-a092-d762defd759f None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:08.434965 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e785e2-82e3-4f33-8b67-4d80de878d32 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:08.476557 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a931bddb-23e3-4df1-8863-ad2c4d7de66e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:08.477207 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a931bddb-23e3-4df1-8863-ad2c4d7de66e None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:08.477540 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a931bddb-23e3-4df1-8863-ad2c4d7de66e None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:08.478071 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a931bddb-23e3-4df1-8863-ad2c4d7de66e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:08.492048 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a931bddb-23e3-4df1-8863-ad2c4d7de66e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:08.493836 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a931bddb-23e3-4df1-8863-ad2c4d7de66e None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:08.510981 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-a931bddb-23e3-4df1-8863-ad2c4d7de66e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:08.510981 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:08.510981 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:08.510981 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:08.510981 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:08.510981 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:08.510981 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:08.510981 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:08.510981 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:08.510981 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:08.510981 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:08.510981 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:08.510981 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:18:08.510981 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:18:08.510981 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:18:08.510981 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:18:08.513120 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:08.513120 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:08.513120 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:08.513120 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:08.513120 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:08.513120 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:08.513120 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:08.513120 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:08.513120 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:08.513120 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:08.513120 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:08.513120 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:08.513120 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:08.513120 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:08.513120 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:08.513120 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:08.513120 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:18:08.515218 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:18:08.515218 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:08.515218 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:18:08.515218 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 244/468] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:18:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:08.515218 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a931bddb-23e3-4df1-8863-ad2c4d7de66e None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:08.553506 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67ac2ff-6142-4310-a092-d762defd759f None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:08.592084 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d6ece1-8357-4504-84b3-106485f67c35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:08.592865 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d6ece1-8357-4504-84b3-106485f67c35 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:08.594136 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d6ece1-8357-4504-84b3-106485f67c35 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:08.594302 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:08.594473 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d6ece1-8357-4504-84b3-106485f67c35 None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:08.594663 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d6ece1-8357-4504-84b3-106485f67c35 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:08.594980 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d6ece1-8357-4504-84b3-106485f67c35 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:08.596518 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d6ece1-8357-4504-84b3-106485f67c35 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:08.600873 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:08.615821 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 225/469] 10.222.0.102 () {56 vars in 1114 bytes} [Thu Sep 1 17:18:08 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:08.617033 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d6ece1-8357-4504-84b3-106485f67c35 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:08.647340 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a931bddb-23e3-4df1-8863-ad2c4d7de66e None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:08.697871 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9d5e2d-eebb-42f3-bb39-90dbeb91115d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:08.698511 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9d5e2d-eebb-42f3-bb39-90dbeb91115d None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:08.698796 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9d5e2d-eebb-42f3-bb39-90dbeb91115d None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:08.699072 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9d5e2d-eebb-42f3-bb39-90dbeb91115d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:08.713685 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9d5e2d-eebb-42f3-bb39-90dbeb91115d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:08.715045 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9d5e2d-eebb-42f3-bb39-90dbeb91115d None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:08.731475 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-9e9d5e2d-eebb-42f3-bb39-90dbeb91115d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:08.731475 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:08.731475 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:08.731475 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:08.731475 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:08.731475 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:08.731475 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:08.731475 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:08.731475 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:08.731475 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:08.731475 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:08.731475 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:08.731475 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:18:08.731475 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:18:08.731475 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:18:08.731475 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:18:08.732585 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:08.732585 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:08.732585 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:08.732585 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:08.732585 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:08.732585 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:08.732585 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:08.732585 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:08.732585 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:08.732585 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:08.732585 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:08.732585 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:08.732585 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:08.732585 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:08.732585 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:08.732585 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:08.732585 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:18:08.733619 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:18:08.733619 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:08.733619 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:18:08.733619 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 245/470] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:18:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:08.733848 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9d5e2d-eebb-42f3-bb39-90dbeb91115d None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:08.744122 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d6ece1-8357-4504-84b3-106485f67c35 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:08.782155 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b971688e-46c4-4860-a987-7b1c3e9a17c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:08.783040 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b971688e-46c4-4860-a987-7b1c3e9a17c5 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:08.783457 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b971688e-46c4-4860-a987-7b1c3e9a17c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:08.783861 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b971688e-46c4-4860-a987-7b1c3e9a17c5 None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:08.784727 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b971688e-46c4-4860-a987-7b1c3e9a17c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:08.785235 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b971688e-46c4-4860-a987-7b1c3e9a17c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:08.786898 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b971688e-46c4-4860-a987-7b1c3e9a17c5 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:08.800473 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 226/471] 10.222.0.102 () {56 vars in 1114 bytes} [Thu Sep 1 17:18:08 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:08.802123 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b971688e-46c4-4860-a987-7b1c3e9a17c5 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:08.850056 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9d5e2d-eebb-42f3-bb39-90dbeb91115d None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:08.866510 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:08.871759 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:08.890404 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27642b11-e65c-4be7-a959-d7cccb8929b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:08.891021 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27642b11-e65c-4be7-a959-d7cccb8929b5 None admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:08.891302 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27642b11-e65c-4be7-a959-d7cccb8929b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:08.891571 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27642b11-e65c-4be7-a959-d7cccb8929b5 None admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:08.892588 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27642b11-e65c-4be7-a959-d7cccb8929b5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:08.894871 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27642b11-e65c-4be7-a959-d7cccb8929b5 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:08.916908 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b971688e-46c4-4860-a987-7b1c3e9a17c5 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:08.925891 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 246/472] 10.222.0.102 () {60 vars in 1137 bytes} [Thu Sep 1 17:18:08 2022] POST /identity/v3/users => generated 249 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:18:08.925891 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27642b11-e65c-4be7-a959-d7cccb8929b5 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:08.963339 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7e783e-f213-4909-ac52-e4805f354794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:08.964264 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7e783e-f213-4909-ac52-e4805f354794 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:08.965442 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7e783e-f213-4909-ac52-e4805f354794 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:08.965932 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7e783e-f213-4909-ac52-e4805f354794 None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:08.966736 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7e783e-f213-4909-ac52-e4805f354794 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:08.967161 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7e783e-f213-4909-ac52-e4805f354794 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:08.968687 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7e783e-f213-4909-ac52-e4805f354794 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:08.997164 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 227/473] 10.222.0.102 () {56 vars in 1114 bytes} [Thu Sep 1 17:18:08 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:08.999283 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cda8a7a-ad41-4659-9fbd-066814ae0cb3 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:08.999999 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cda8a7a-ad41-4659-9fbd-066814ae0cb3 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:09.000849 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cda8a7a-ad41-4659-9fbd-066814ae0cb3 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:09.004416 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 228/474] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:18:08 2022] GET /identity => generated 271 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:18:09.006572 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cda8a7a-ad41-4659-9fbd-066814ae0cb3 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:09.069671 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27642b11-e65c-4be7-a959-d7cccb8929b5 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:09.115693 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ae6f01-7b88-462a-949a-54c84865b068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:09.116454 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ae6f01-7b88-462a-949a-54c84865b068 None admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:09.116754 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ae6f01-7b88-462a-949a-54c84865b068 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:09.117131 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ae6f01-7b88-462a-949a-54c84865b068 None admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:09.118107 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ae6f01-7b88-462a-949a-54c84865b068 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:09.119989 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ae6f01-7b88-462a-949a-54c84865b068 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:09.146142 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cda8a7a-ad41-4659-9fbd-066814ae0cb3 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:09.148753 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 247/475] 10.222.0.102 () {60 vars in 1137 bytes} [Thu Sep 1 17:18:08 2022] POST /identity/v3/users => generated 249 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:18:09.151235 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ea5749-3804-444a-8954-db238a4a4963 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:09.151520 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ea5749-3804-444a-8954-db238a4a4963 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:09.151792 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ea5749-3804-444a-8954-db238a4a4963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:09.172324 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-f8ea5749-3804-444a-8954-db238a4a4963 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:18:09.173650 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-f8ea5749-3804-444a-8954-db238a4a4963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:18:09.192322 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:09.207620 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f6fe82-9e52-4625-9815-273132b93e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:09.209024 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:09.209725 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f6fe82-9e52-4625-9815-273132b93e67 None admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:09.211411 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f6fe82-9e52-4625-9815-273132b93e67 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:09.212588 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f6fe82-9e52-4625-9815-273132b93e67 None admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:09.214853 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f6fe82-9e52-4625-9815-273132b93e67 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:09.218602 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f6fe82-9e52-4625-9815-273132b93e67 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:09.251996 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 229/476] 10.222.0.102 () {60 vars in 1137 bytes} [Thu Sep 1 17:18:09 2022] POST /identity/v3/users => generated 250 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:18:09.286230 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 248/477] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:18:09 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:18:09.574501 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f6fe82-9e52-4625-9815-273132b93e67 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:09.705205 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f6fe82-9e52-4625-9815-273132b93e67 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:09.758281 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e419b7d-d04b-4763-a07c-851e9b9ef70c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:09.758905 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e419b7d-d04b-4763-a07c-851e9b9ef70c None admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:09.759191 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e419b7d-d04b-4763-a07c-851e9b9ef70c None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:09.759470 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e419b7d-d04b-4763-a07c-851e9b9ef70c None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:09.760155 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e419b7d-d04b-4763-a07c-851e9b9ef70c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:09.761587 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e419b7d-d04b-4763-a07c-851e9b9ef70c None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:09.775432 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.sql.core [None req-2e419b7d-d04b-4763-a07c-851e9b9ef70c None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 01 17:18:09.775432 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 01 17:18:09.775432 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [parameters: {'id': '20169541ebdc43f3930055618d80833f', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 01 17:18:09.775432 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81502) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 01 17:18:09.796031 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-2e419b7d-d04b-4763-a07c-851e9b9ef70c None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 01 17:18:09.796031 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:09.796031 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 01 17:18:09.796031 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 01 17:18:09.796031 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 01 17:18:09.796031 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 01 17:18:09.796031 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 01 17:18:09.796031 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application result = self._query(query) Sep 01 17:18:09.796031 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 01 17:18:09.796031 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application conn.query(q) Sep 01 17:18:09.796031 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 01 17:18:09.796031 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 01 17:18:09.796031 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 01 17:18:09.796031 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application result.read() Sep 01 17:18:09.796031 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 01 17:18:09.796903 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 01 17:18:09.796903 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 01 17:18:09.796903 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 01 17:18:09.796903 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 01 17:18:09.796903 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 01 17:18:09.796903 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 01 17:18:09.796903 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 01 17:18:09.796903 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 01 17:18:09.796903 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:18:09.796903 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 01 17:18:09.796903 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:18:09.796903 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:09.796903 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 01 17:18:09.796903 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 01 17:18:09.796903 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 01 17:18:09.796903 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return ref.to_dict() Sep 01 17:18:09.796903 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 01 17:18:09.796903 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application next(self.gen) Sep 01 17:18:09.797814 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 01 17:18:09.797814 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application yield resource Sep 01 17:18:09.797814 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 01 17:18:09.797814 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application next(self.gen) Sep 01 17:18:09.797814 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 01 17:18:09.797814 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self.session.rollback() Sep 01 17:18:09.797814 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 01 17:18:09.797814 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self.force_reraise() Sep 01 17:18:09.797814 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 01 17:18:09.797814 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise self.value Sep 01 17:18:09.797814 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 01 17:18:09.797814 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 01 17:18:09.797814 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 01 17:18:09.797814 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application session.commit() Sep 01 17:18:09.797814 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 01 17:18:09.797814 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 01 17:18:09.797814 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 01 17:18:09.798639 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._prepare_impl() Sep 01 17:18:09.798639 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 01 17:18:09.798639 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self.session.flush() Sep 01 17:18:09.798639 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 01 17:18:09.798639 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._flush(objects) Sep 01 17:18:09.798639 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 01 17:18:09.798639 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 01 17:18:09.798639 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 01 17:18:09.798639 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application compat.raise_( Sep 01 17:18:09.798639 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 01 17:18:09.798639 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception Sep 01 17:18:09.798639 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 01 17:18:09.798639 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application flush_context.execute() Sep 01 17:18:09.798639 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 01 17:18:09.798639 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rec.execute(self) Sep 01 17:18:09.798639 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 01 17:18:09.798639 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 01 17:18:09.799504 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 01 17:18:09.799504 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 01 17:18:09.799504 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 01 17:18:09.799504 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 01 17:18:09.799504 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 01 17:18:09.799504 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 01 17:18:09.799504 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 01 17:18:09.799504 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 01 17:18:09.799504 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 01 17:18:09.799504 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 01 17:18:09.799504 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 01 17:18:09.799504 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 01 17:18:09.799504 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 01 17:18:09.799504 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 01 17:18:09.799504 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 01 17:18:09.799504 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception Sep 01 17:18:09.800323 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 01 17:18:09.800323 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 01 17:18:09.800323 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 01 17:18:09.800323 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 01 17:18:09.800323 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 01 17:18:09.800323 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application result = self._query(query) Sep 01 17:18:09.800323 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 01 17:18:09.800323 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application conn.query(q) Sep 01 17:18:09.800323 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 01 17:18:09.800323 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 01 17:18:09.800323 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 01 17:18:09.800323 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application result.read() Sep 01 17:18:09.800323 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 01 17:18:09.800323 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 01 17:18:09.800323 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 01 17:18:09.800323 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 01 17:18:09.802800 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 01 17:18:09.802800 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 01 17:18:09.802800 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 01 17:18:09.802800 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 01 17:18:09.802800 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 01 17:18:09.802800 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 01 17:18:09.802800 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [parameters: {'id': '20169541ebdc43f3930055618d80833f', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 01 17:18:09.802800 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 01 17:18:09.802800 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:18:09.802800 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 17:18:09.802800 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:18:09.802800 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:09.802800 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:09.802800 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:09.802800 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:09.802800 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:09.804784 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:09.804784 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:09.804784 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:09.804784 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:09.804784 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:09.804784 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:09.804784 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 01 17:18:09.804784 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 01 17:18:09.804784 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:09.804784 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:09.804784 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 01 17:18:09.804784 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 01 17:18:09.804784 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 01 17:18:09.804784 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 01 17:18:09.804784 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 01 17:18:09.804784 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:18:09.806844 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 230/478] 10.222.0.102 () {60 vars in 1136 bytes} [Thu Sep 1 17:18:09 2022] POST /identity/v3/roles => generated 144 bytes in 224 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:09.806985 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ea5749-3804-444a-8954-db238a4a4963 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:09.960031 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ea5749-3804-444a-8954-db238a4a4963 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:10.007103 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5ab4fc-7526-4474-9577-e70533d60290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:10.007738 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5ab4fc-7526-4474-9577-e70533d60290 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:10.008019 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5ab4fc-7526-4474-9577-e70533d60290 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:10.008293 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5ab4fc-7526-4474-9577-e70533d60290 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:10.021319 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5ab4fc-7526-4474-9577-e70533d60290 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:18:10.026138 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-ca5ab4fc-7526-4474-9577-e70533d60290 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 17:18:10.026138 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:10.026138 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:10.026138 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:10.026138 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:10.026138 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:10.026138 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:10.026138 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:10.026138 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:10.026138 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:10.026138 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:10.026138 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:10.026138 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:18:10.026138 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:18:10.026138 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:18:10.026138 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:18:10.027018 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:18:10.027018 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:18:10.027018 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:18:10.027018 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:18:10.027018 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:10.027018 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:10.027018 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:10.027018 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:10.027018 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:10.027018 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:10.027018 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:10.027018 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:10.027018 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:10.027018 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:10.027018 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:10.027018 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:10.027796 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:10.027796 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:10.027796 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:10.027796 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:10.027796 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:18:10.027796 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:18:10.027796 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:18:10.027796 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:18:10.027796 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:18:10.027796 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:18:10.027796 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 17:18:10.027796 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:18:10.031975 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 249/479] 10.222.0.102 () {56 vars in 1119 bytes} [Thu Sep 1 17:18:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:10.033339 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e419b7d-d04b-4763-a07c-851e9b9ef70c None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:10.080400 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:10.154749 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e419b7d-d04b-4763-a07c-851e9b9ef70c None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:10.190610 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1386f7c-7526-4988-93a4-00743044c3d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:10.191227 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1386f7c-7526-4988-93a4-00743044c3d8 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:10.191511 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1386f7c-7526-4988-93a4-00743044c3d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:10.192348 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1386f7c-7526-4988-93a4-00743044c3d8 None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:10.193014 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1386f7c-7526-4988-93a4-00743044c3d8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:10.193334 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1386f7c-7526-4988-93a4-00743044c3d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:10.194786 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1386f7c-7526-4988-93a4-00743044c3d8 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:10.209503 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 231/480] 10.222.0.102 () {56 vars in 1116 bytes} [Thu Sep 1 17:18:10 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:18:10.683849 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402e7654-a6cf-408e-a2a6-259f9ded8257 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:10.684360 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402e7654-a6cf-408e-a2a6-259f9ded8257 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:10.684360 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402e7654-a6cf-408e-a2a6-259f9ded8257 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:10.685998 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 250/481] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:18:10 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:18:10.691383 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2533758-4743-47be-902a-1106181851f9 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:10.692340 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2533758-4743-47be-902a-1106181851f9 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:10.692551 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2533758-4743-47be-902a-1106181851f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:10.711463 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-c2533758-4743-47be-902a-1106181851f9 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:18:10.712705 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-c2533758-4743-47be-902a-1106181851f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:18:10.811937 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 232/482] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:18:10 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 17:18:11.268862 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402e7654-a6cf-408e-a2a6-259f9ded8257 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:11.292659 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:11.404549 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dd7154-ab21-4e3d-a995-bd2cad67d856 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:11.404761 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402e7654-a6cf-408e-a2a6-259f9ded8257 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:11.405255 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dd7154-ab21-4e3d-a995-bd2cad67d856 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:11.405762 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dd7154-ab21-4e3d-a995-bd2cad67d856 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:11.408548 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 233/483] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:18:11 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:18:11.413775 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704842d3-b776-4ff4-aeb4-e9cbdededc51 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:11.414341 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704842d3-b776-4ff4-aeb4-e9cbdededc51 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:11.414902 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704842d3-b776-4ff4-aeb4-e9cbdededc51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:11.434706 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-704842d3-b776-4ff4-aeb4-e9cbdededc51 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:18:11.436342 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-704842d3-b776-4ff4-aeb4-e9cbdededc51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:18:11.456130 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d197da8-39fd-43e4-91db-5b23e683f7b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:11.456759 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d197da8-39fd-43e4-91db-5b23e683f7b6 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:11.457036 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d197da8-39fd-43e4-91db-5b23e683f7b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:11.457846 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d197da8-39fd-43e4-91db-5b23e683f7b6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:11.475970 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:11.482223 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d197da8-39fd-43e4-91db-5b23e683f7b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:18:11.489283 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-6d197da8-39fd-43e4-91db-5b23e683f7b6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:18:11.489283 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:11.489283 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:11.489283 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:11.489283 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:11.489283 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:11.489283 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:11.489283 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:11.489283 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:11.489283 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:11.489283 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:11.489283 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:11.489283 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:18:11.489283 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:18:11.489283 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:18:11.489283 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:18:11.490185 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:18:11.490185 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:18:11.490185 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:18:11.490185 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:18:11.490185 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:11.490185 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:11.490185 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:11.490185 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:11.490185 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:11.490185 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:11.490185 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:11.490185 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:11.490185 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:11.490185 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:11.490185 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:11.490185 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:11.490960 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:11.490960 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:11.490960 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:11.490960 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:11.490960 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:18:11.490960 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:18:11.490960 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:18:11.490960 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:18:11.490960 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:18:11.490960 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:18:11.490960 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:18:11.490960 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:18:11.490960 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 251/484] 10.222.0.102 () {56 vars in 1107 bytes} [Thu Sep 1 17:18:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:11.496861 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d197da8-39fd-43e4-91db-5b23e683f7b6 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:11.561787 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 234/485] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:18:11 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 17:18:11.625477 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:11.649736 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d197da8-39fd-43e4-91db-5b23e683f7b6 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:11.695257 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fc1fac-9313-475e-93f9-c75d2e320c51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:11.696232 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fc1fac-9313-475e-93f9-c75d2e320c51 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:11.696232 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fc1fac-9313-475e-93f9-c75d2e320c51 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:11.696499 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fc1fac-9313-475e-93f9-c75d2e320c51 None admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:11.697044 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fc1fac-9313-475e-93f9-c75d2e320c51 None admin] RBAC: Adding query filter params (name=service) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:11.697372 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fc1fac-9313-475e-93f9-c75d2e320c51 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:11.698737 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fc1fac-9313-475e-93f9-c75d2e320c51 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:11.712977 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 252/486] 10.222.0.102 () {56 vars in 1108 bytes} [Thu Sep 1 17:18:11 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:11.713985 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704842d3-b776-4ff4-aeb4-e9cbdededc51 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:11.722087 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2135d282-178d-4ed5-b79d-330bf05fea7c None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:11.722382 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2135d282-178d-4ed5-b79d-330bf05fea7c None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:11.722567 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2135d282-178d-4ed5-b79d-330bf05fea7c None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:11.724429 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 253/487] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:18:11 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Sep 01 17:18:11.730119 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c3313e-9610-4679-b0e2-046af5b46412 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:11.730698 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c3313e-9610-4679-b0e2-046af5b46412 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:11.730974 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c3313e-9610-4679-b0e2-046af5b46412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:11.764424 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-04c3313e-9610-4679-b0e2-046af5b46412 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:18:11.765578 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-04c3313e-9610-4679-b0e2-046af5b46412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:18:11.871721 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704842d3-b776-4ff4-aeb4-e9cbdededc51 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:11.885691 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 254/488] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:18:11 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 17:18:11.887962 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede4fc06-654d-4ee4-aa07-6752e27bbfca None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:11.888388 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede4fc06-654d-4ee4-aa07-6752e27bbfca None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:11.888903 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede4fc06-654d-4ee4-aa07-6752e27bbfca None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:11.890821 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 255/489] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:18:11 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:18:11.896630 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb1ff01-893f-4e40-8cae-80bf4e058d4e None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:11.896989 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb1ff01-893f-4e40-8cae-80bf4e058d4e None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:11.897389 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb1ff01-893f-4e40-8cae-80bf4e058d4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:11.912747 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109e794d-f7ad-4044-832c-958b2740ec43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:11.913686 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109e794d-f7ad-4044-832c-958b2740ec43 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:11.914170 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109e794d-f7ad-4044-832c-958b2740ec43 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:11.914618 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109e794d-f7ad-4044-832c-958b2740ec43 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:11.917501 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-4bb1ff01-893f-4e40-8cae-80bf4e058d4e None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:18:11.919147 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-4bb1ff01-893f-4e40-8cae-80bf4e058d4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:18:11.929727 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109e794d-f7ad-4044-832c-958b2740ec43 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:11.931488 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109e794d-f7ad-4044-832c-958b2740ec43 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:11.948305 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-109e794d-f7ad-4044-832c-958b2740ec43 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:11.948305 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:11.948305 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:11.948305 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:11.948305 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:11.948305 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:11.948305 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:11.948305 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:11.948305 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:11.948305 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:11.948305 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:11.948305 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:11.948305 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:18:11.948305 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:18:11.948305 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:18:11.948305 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:18:11.949266 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:11.949266 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:11.949266 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:11.949266 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:11.949266 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:11.949266 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:11.949266 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:11.949266 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:11.949266 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:11.949266 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:11.949266 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:11.949266 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:11.949266 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:11.949266 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:11.949266 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:11.949266 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:11.949266 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:18:11.950133 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:18:11.950133 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:11.950133 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:18:11.950133 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 235/490] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:18:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:11.955521 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109e794d-f7ad-4044-832c-958b2740ec43 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:11.972232 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:12.004932 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 256/491] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:18:11 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 17:18:12.006159 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb1ff01-893f-4e40-8cae-80bf4e058d4e None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:12.010228 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:12.142071 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109e794d-f7ad-4044-832c-958b2740ec43 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:12.184845 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb1ff01-893f-4e40-8cae-80bf4e058d4e None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:12.204714 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c7a0f1-4bb4-4d79-8e53-3bcb689dbf06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:12.205626 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c7a0f1-4bb4-4d79-8e53-3bcb689dbf06 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:12.206110 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c7a0f1-4bb4-4d79-8e53-3bcb689dbf06 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:12.206601 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c7a0f1-4bb4-4d79-8e53-3bcb689dbf06 None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:12.207485 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c7a0f1-4bb4-4d79-8e53-3bcb689dbf06 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:12.207985 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c7a0f1-4bb4-4d79-8e53-3bcb689dbf06 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:12.209625 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c7a0f1-4bb4-4d79-8e53-3bcb689dbf06 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:12.224246 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 236/492] 10.222.0.102 () {56 vars in 1114 bytes} [Thu Sep 1 17:18:11 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:12.225362 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c7a0f1-4bb4-4d79-8e53-3bcb689dbf06 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:12.231919 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27234d7c-782c-43ee-8197-6fdfa4f7f1cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:12.232550 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27234d7c-782c-43ee-8197-6fdfa4f7f1cc None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:12.232826 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27234d7c-782c-43ee-8197-6fdfa4f7f1cc None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:12.233098 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27234d7c-782c-43ee-8197-6fdfa4f7f1cc None admin] PATH_INFO: `/v3/roles/service` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:12.244409 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27234d7c-782c-43ee-8197-6fdfa4f7f1cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:18:12.249138 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-27234d7c-782c-43ee-8197-6fdfa4f7f1cc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:18:12.249138 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:12.249138 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:12.249138 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:12.249138 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:12.249138 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:12.249138 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:12.249138 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:12.249138 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:12.249138 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:12.249138 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:12.249138 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:12.249138 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:18:12.249138 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:18:12.249138 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:18:12.249138 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:18:12.250008 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:18:12.250008 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:18:12.250008 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:18:12.250008 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:18:12.250008 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:12.250008 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:12.250008 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:12.250008 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:12.250008 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:12.250008 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:12.250008 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:12.250008 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:12.250008 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:12.250008 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:12.250008 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:12.250008 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:12.250793 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:12.250793 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:12.250793 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:12.250793 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:12.250793 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:18:12.250793 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:18:12.250793 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:18:12.250793 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:18:12.250793 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:18:12.250793 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:18:12.250793 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:18:12.250793 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:18:12.255702 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 257/493] 10.222.0.102 () {56 vars in 1107 bytes} [Thu Sep 1 17:18:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:12.256547 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27234d7c-782c-43ee-8197-6fdfa4f7f1cc None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:12.311058 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:12.338310 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:12.347061 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c7a0f1-4bb4-4d79-8e53-3bcb689dbf06 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:12.381819 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27234d7c-782c-43ee-8197-6fdfa4f7f1cc None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:12.387525 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e21b388-56ce-4d9b-884b-6a63dd30c68c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:12.388165 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e21b388-56ce-4d9b-884b-6a63dd30c68c None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:12.388443 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e21b388-56ce-4d9b-884b-6a63dd30c68c None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:12.388718 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e21b388-56ce-4d9b-884b-6a63dd30c68c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:12.401972 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e21b388-56ce-4d9b-884b-6a63dd30c68c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:18:12.407101 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-9e21b388-56ce-4d9b-884b-6a63dd30c68c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:18:12.407101 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:12.407101 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:12.407101 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:12.407101 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:12.407101 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:12.407101 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:12.407101 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:12.407101 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:12.407101 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:12.407101 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:12.407101 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:12.407101 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:18:12.407101 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:18:12.407101 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:18:12.407101 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:18:12.407970 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:18:12.407970 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:18:12.407970 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:18:12.407970 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:18:12.407970 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:12.407970 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:12.407970 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:12.407970 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:12.407970 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:12.407970 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:12.407970 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:12.407970 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:12.407970 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:12.407970 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:12.407970 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:12.407970 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:12.408739 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:12.408739 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:12.408739 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:12.408739 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:12.408739 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:18:12.408739 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:18:12.408739 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:18:12.408739 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:18:12.408739 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:18:12.408739 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:18:12.408739 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:18:12.408739 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:18:12.413432 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 237/494] 10.222.0.102 () {56 vars in 1107 bytes} [Thu Sep 1 17:18:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:12.413432 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e21b388-56ce-4d9b-884b-6a63dd30c68c None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:12.420420 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7208a3d2-8626-457d-9b4b-883bb990406b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:12.421051 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7208a3d2-8626-457d-9b4b-883bb990406b None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:12.421357 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7208a3d2-8626-457d-9b4b-883bb990406b None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:12.421708 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7208a3d2-8626-457d-9b4b-883bb990406b None admin] PATH_INFO: `/v3/users/nova` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:12.430356 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7208a3d2-8626-457d-9b4b-883bb990406b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:12.431887 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7208a3d2-8626-457d-9b4b-883bb990406b None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:12.445831 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-7208a3d2-8626-457d-9b4b-883bb990406b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 17:18:12.445831 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:12.445831 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:12.445831 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:12.445831 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:12.445831 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:12.445831 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:12.445831 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:12.445831 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:12.445831 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:12.445831 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:12.445831 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:12.445831 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:18:12.445831 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:18:12.445831 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:18:12.445831 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:18:12.446804 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:12.446804 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:12.446804 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:18:12.446804 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:18:12.446804 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:18:12.446804 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:18:12.446804 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:12.446804 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:12.446804 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:12.446804 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:12.446804 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:12.446804 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:12.446804 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:12.446804 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:12.446804 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:12.446804 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:12.446804 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:12.446804 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:12.447696 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:12.447696 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:12.447696 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:18:12.447696 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:18:12.447696 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:18:12.447696 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:18:12.447696 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:18:12.447696 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:18:12.447696 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 17:18:12.447696 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:18:12.451730 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 258/495] 10.222.0.102 () {56 vars in 1098 bytes} [Thu Sep 1 17:18:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:12.452377 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7208a3d2-8626-457d-9b4b-883bb990406b None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:12.532656 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e21b388-56ce-4d9b-884b-6a63dd30c68c None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:12.562381 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7208a3d2-8626-457d-9b4b-883bb990406b None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:12.565917 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:12.573417 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff70ca0-49ec-43c8-86d2-c30e4c92af24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:12.574143 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff70ca0-49ec-43c8-86d2-c30e4c92af24 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:12.574515 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff70ca0-49ec-43c8-86d2-c30e4c92af24 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:12.574877 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff70ca0-49ec-43c8-86d2-c30e4c92af24 None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:12.575644 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff70ca0-49ec-43c8-86d2-c30e4c92af24 None admin] RBAC: Adding query filter params (name=service) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:12.576060 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff70ca0-49ec-43c8-86d2-c30e4c92af24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:12.577616 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff70ca0-49ec-43c8-86d2-c30e4c92af24 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:12.585235 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:12.588566 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 238/496] 10.222.0.102 () {56 vars in 1108 bytes} [Thu Sep 1 17:18:12 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:12.593553 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff70ca0-49ec-43c8-86d2-c30e4c92af24 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:12.599180 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe25034-0460-425f-a729-7753838300bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:12.599881 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe25034-0460-425f-a729-7753838300bb None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:12.600229 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe25034-0460-425f-a729-7753838300bb None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:12.600566 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe25034-0460-425f-a729-7753838300bb None admin] PATH_INFO: `/v3/roles/service` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:12.612730 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe25034-0460-425f-a729-7753838300bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:18:12.618688 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-6fe25034-0460-425f-a729-7753838300bb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:18:12.618688 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:12.618688 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:12.618688 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:12.618688 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:12.618688 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:12.618688 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:12.618688 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:12.618688 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:12.618688 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:12.618688 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:12.618688 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:12.618688 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:18:12.618688 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:18:12.618688 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:18:12.618688 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:18:12.619747 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:18:12.619747 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:18:12.619747 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:18:12.619747 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:18:12.619747 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:12.619747 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:12.619747 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:12.619747 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:12.619747 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:12.619747 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:12.619747 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:12.619747 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:12.619747 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:12.619747 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:12.619747 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:12.619747 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:12.620759 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:12.620759 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:12.620759 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:12.620759 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:12.620759 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:18:12.620759 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:18:12.620759 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:18:12.620759 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:18:12.620759 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:18:12.620759 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:18:12.620759 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:18:12.620759 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:18:12.621470 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 259/497] 10.222.0.102 () {56 vars in 1107 bytes} [Thu Sep 1 17:18:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:12.622158 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe25034-0460-425f-a729-7753838300bb None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:12.718636 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff70ca0-49ec-43c8-86d2-c30e4c92af24 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:12.758923 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070c5993-f32e-4fff-8230-632bc19e9e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:12.759560 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070c5993-f32e-4fff-8230-632bc19e9e3a None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:12.759837 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070c5993-f32e-4fff-8230-632bc19e9e3a None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:12.760112 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070c5993-f32e-4fff-8230-632bc19e9e3a None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:12.760431 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe25034-0460-425f-a729-7753838300bb None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:12.760859 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070c5993-f32e-4fff-8230-632bc19e9e3a None admin] RBAC: Adding query filter params (name=service) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:12.761155 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070c5993-f32e-4fff-8230-632bc19e9e3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:12.762612 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070c5993-f32e-4fff-8230-632bc19e9e3a None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:12.772493 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 239/498] 10.222.0.102 () {56 vars in 1108 bytes} [Thu Sep 1 17:18:12 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:12.773367 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070c5993-f32e-4fff-8230-632bc19e9e3a None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:12.806788 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c59b88d-e95c-4932-90ea-77a4e223276e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:12.807709 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c59b88d-e95c-4932-90ea-77a4e223276e None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:12.808173 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c59b88d-e95c-4932-90ea-77a4e223276e None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:12.808443 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c59b88d-e95c-4932-90ea-77a4e223276e None admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:12.809187 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c59b88d-e95c-4932-90ea-77a4e223276e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:12.809517 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c59b88d-e95c-4932-90ea-77a4e223276e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:12.811002 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c59b88d-e95c-4932-90ea-77a4e223276e None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:12.843528 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:12.856415 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 260/499] 10.222.0.102 () {56 vars in 1138 bytes} [Thu Sep 1 17:18:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 370 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:12.857543 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c59b88d-e95c-4932-90ea-77a4e223276e None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:12.897235 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070c5993-f32e-4fff-8230-632bc19e9e3a None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:12.899634 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:12.931604 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250d4c19-0ee0-40b5-97a7-3c53810cf435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:12.932233 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250d4c19-0ee0-40b5-97a7-3c53810cf435 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:12.932518 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250d4c19-0ee0-40b5-97a7-3c53810cf435 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:12.932786 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250d4c19-0ee0-40b5-97a7-3c53810cf435 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:12.944429 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250d4c19-0ee0-40b5-97a7-3c53810cf435 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:12.945951 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250d4c19-0ee0-40b5-97a7-3c53810cf435 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:12.958104 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c59b88d-e95c-4932-90ea-77a4e223276e None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:12.960320 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-250d4c19-0ee0-40b5-97a7-3c53810cf435 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:12.960320 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:12.960320 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:12.960320 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:12.960320 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:12.960320 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:12.960320 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:12.960320 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:12.960320 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:12.960320 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:12.960320 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:12.960320 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:12.960320 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:18:12.960320 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:18:12.960320 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:18:12.960320 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:18:12.961177 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:12.961177 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:12.961177 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:12.961177 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:12.961177 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:12.961177 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:12.961177 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:12.961177 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:12.961177 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:12.961177 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:12.961177 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:12.961177 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:12.961177 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:12.961177 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:12.961177 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:12.961177 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:12.961177 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:18:12.962071 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:18:12.962071 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:12.962071 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:18:12.962071 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 240/500] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:18:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:12.962289 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250d4c19-0ee0-40b5-97a7-3c53810cf435 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:12.993389 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f3056b-d1a4-4ddf-bd91-0dea97d3bfac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:12.993919 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f3056b-d1a4-4ddf-bd91-0dea97d3bfac None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:12.994197 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f3056b-d1a4-4ddf-bd91-0dea97d3bfac None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:12.994467 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f3056b-d1a4-4ddf-bd91-0dea97d3bfac None admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:12.995130 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f3056b-d1a4-4ddf-bd91-0dea97d3bfac None admin] RBAC: Adding query filter params (name=service) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:12.995433 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f3056b-d1a4-4ddf-bd91-0dea97d3bfac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:12.996806 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f3056b-d1a4-4ddf-bd91-0dea97d3bfac None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:13.005881 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 261/501] 10.222.0.102 () {56 vars in 1108 bytes} [Thu Sep 1 17:18:12 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:13.007107 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4993197b-f03f-48d3-879f-25c32da0e8c4 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:13.007320 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4993197b-f03f-48d3-879f-25c32da0e8c4 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:13.007598 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4993197b-f03f-48d3-879f-25c32da0e8c4 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:13.012395 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 262/502] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:18:13 2022] GET /identity => generated 271 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:18:13.012395 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4993197b-f03f-48d3-879f-25c32da0e8c4 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:13.070011 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250d4c19-0ee0-40b5-97a7-3c53810cf435 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:13.080715 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:13.107912 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d00ab6e-9806-4910-ae30-d52ed51430f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:13.108538 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d00ab6e-9806-4910-ae30-d52ed51430f9 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:13.108819 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d00ab6e-9806-4910-ae30-d52ed51430f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:13.109087 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d00ab6e-9806-4910-ae30-d52ed51430f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:13.120944 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d00ab6e-9806-4910-ae30-d52ed51430f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:13.122466 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d00ab6e-9806-4910-ae30-d52ed51430f9 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:13.127893 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4993197b-f03f-48d3-879f-25c32da0e8c4 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:13.130130 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:13.137704 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-6d00ab6e-9806-4910-ae30-d52ed51430f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:13.137704 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:13.137704 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:13.137704 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:13.137704 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:13.137704 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:13.137704 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:13.137704 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:13.137704 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:13.137704 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:13.137704 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:13.137704 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:13.137704 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:18:13.137704 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:18:13.137704 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:18:13.137704 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:18:13.138837 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:13.138837 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:13.138837 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:13.138837 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:13.138837 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:13.138837 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:13.138837 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:13.138837 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:13.138837 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:13.138837 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:13.138837 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:13.138837 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:13.138837 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:13.138837 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:13.138837 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:13.138837 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:13.138837 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:18:13.139690 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:18:13.139690 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:13.139690 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:18:13.139690 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 241/503] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:18:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:13.140022 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d00ab6e-9806-4910-ae30-d52ed51430f9 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:13.170727 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f1bbdb-7591-4725-9f5e-4aabcff672e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:13.171416 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f1bbdb-7591-4725-9f5e-4aabcff672e5 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:13.171743 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f1bbdb-7591-4725-9f5e-4aabcff672e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:13.171983 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f1bbdb-7591-4725-9f5e-4aabcff672e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:13.241080 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f1bbdb-7591-4725-9f5e-4aabcff672e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:13.242580 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f1bbdb-7591-4725-9f5e-4aabcff672e5 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:13.243470 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d00ab6e-9806-4910-ae30-d52ed51430f9 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:13.279115 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01103fd3-e90b-44bd-862b-5dcf8dbcc142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:13.279723 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01103fd3-e90b-44bd-862b-5dcf8dbcc142 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:13.280004 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01103fd3-e90b-44bd-862b-5dcf8dbcc142 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:13.280273 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01103fd3-e90b-44bd-862b-5dcf8dbcc142 None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:13.280916 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01103fd3-e90b-44bd-862b-5dcf8dbcc142 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:13.281218 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01103fd3-e90b-44bd-862b-5dcf8dbcc142 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:13.282628 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01103fd3-e90b-44bd-862b-5dcf8dbcc142 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:13.296538 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 242/504] 10.222.0.102 () {56 vars in 1114 bytes} [Thu Sep 1 17:18:13 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:13.297428 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01103fd3-e90b-44bd-862b-5dcf8dbcc142 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:13.317004 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:13.359697 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 263/505] 10.222.0.102 () {58 vars in 1287 bytes} [Thu Sep 1 17:18:13 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 17:18:13.361631 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cccf18d-7845-4c1b-a25d-320849745b48 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:13.361992 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cccf18d-7845-4c1b-a25d-320849745b48 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:13.362352 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cccf18d-7845-4c1b-a25d-320849745b48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:13.369498 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:13.383241 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-7cccf18d-7845-4c1b-a25d-320849745b48 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:18:13.384458 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-7cccf18d-7845-4c1b-a25d-320849745b48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:18:13.415262 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01103fd3-e90b-44bd-862b-5dcf8dbcc142 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:13.452627 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeff307-dabf-4fad-9919-4af29af9afde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:13.453256 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeff307-dabf-4fad-9919-4af29af9afde None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:13.453560 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeff307-dabf-4fad-9919-4af29af9afde None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:13.453833 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeff307-dabf-4fad-9919-4af29af9afde None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:13.453918 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 264/506] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:18:13 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:18:13.454701 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cccf18d-7845-4c1b-a25d-320849745b48 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:13.465111 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beeff307-dabf-4fad-9919-4af29af9afde None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:13.466953 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beeff307-dabf-4fad-9919-4af29af9afde None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:13.483806 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-beeff307-dabf-4fad-9919-4af29af9afde None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:13.483806 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:13.483806 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:13.483806 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:13.483806 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:13.483806 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:13.483806 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:13.483806 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:13.483806 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:13.483806 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:13.483806 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:13.483806 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:13.483806 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:18:13.483806 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:18:13.483806 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:18:13.483806 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:18:13.484927 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:13.484927 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:13.484927 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:13.484927 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:13.484927 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:13.484927 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:13.484927 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:13.484927 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:13.484927 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:13.484927 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:13.484927 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:13.484927 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:13.484927 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:13.484927 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:13.484927 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:13.484927 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:13.484927 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:18:13.486153 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:18:13.486153 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:13.486153 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:18:13.491924 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 243/507] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:18:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:13.492549 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeff307-dabf-4fad-9919-4af29af9afde None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:13.566598 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:13.566811 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cccf18d-7845-4c1b-a25d-320849745b48 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:13.598406 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:13.606101 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3ef9ad-73cc-4001-8730-a7d982c80d25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:13.608028 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeff307-dabf-4fad-9919-4af29af9afde None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:13.610030 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3ef9ad-73cc-4001-8730-a7d982c80d25 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:13.610423 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3ef9ad-73cc-4001-8730-a7d982c80d25 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:13.610800 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3ef9ad-73cc-4001-8730-a7d982c80d25 None admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:13.611567 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3ef9ad-73cc-4001-8730-a7d982c80d25 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:13.611977 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3ef9ad-73cc-4001-8730-a7d982c80d25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:13.613519 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3ef9ad-73cc-4001-8730-a7d982c80d25 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:13.630902 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 265/508] 10.222.0.102 () {56 vars in 1114 bytes} [Thu Sep 1 17:18:13 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:13.630902 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3ef9ad-73cc-4001-8730-a7d982c80d25 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:13.647417 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588f6201-9e2c-4539-b8e0-f5818385e81f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:13.648296 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588f6201-9e2c-4539-b8e0-f5818385e81f None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:13.648791 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588f6201-9e2c-4539-b8e0-f5818385e81f None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:13.649362 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588f6201-9e2c-4539-b8e0-f5818385e81f None admin] PATH_INFO: `/v3/users/glance` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:13.659282 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588f6201-9e2c-4539-b8e0-f5818385e81f None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:13.661545 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588f6201-9e2c-4539-b8e0-f5818385e81f None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:13.678536 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-588f6201-9e2c-4539-b8e0-f5818385e81f None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 01 17:18:13.678536 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:13.678536 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:13.678536 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:13.678536 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:13.678536 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:13.678536 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:13.678536 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:13.678536 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:13.678536 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:13.678536 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:13.678536 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:13.678536 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:18:13.678536 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:18:13.678536 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:18:13.678536 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:18:13.679621 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:13.679621 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:13.679621 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:18:13.679621 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:18:13.679621 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:18:13.679621 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:18:13.679621 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:13.679621 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:13.679621 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:13.679621 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:13.679621 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:13.679621 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:13.679621 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:13.679621 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:13.679621 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:13.679621 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:13.679621 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:13.679621 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:13.680653 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:13.680653 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:13.680653 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:18:13.680653 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:18:13.680653 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:18:13.680653 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:18:13.680653 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:18:13.680653 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:18:13.680653 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 01 17:18:13.680653 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:18:13.680653 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 244/509] 10.222.0.102 () {56 vars in 1104 bytes} [Thu Sep 1 17:18:13 2022] GET /identity/v3/users/glance => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:13.684048 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588f6201-9e2c-4539-b8e0-f5818385e81f None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:13.745510 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3ef9ad-73cc-4001-8730-a7d982c80d25 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:13.779527 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92032b72-c5c8-4544-9690-8e934934f336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:13.780166 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92032b72-c5c8-4544-9690-8e934934f336 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:13.780410 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92032b72-c5c8-4544-9690-8e934934f336 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:13.780684 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92032b72-c5c8-4544-9690-8e934934f336 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:13.786492 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588f6201-9e2c-4539-b8e0-f5818385e81f None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:13.792056 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92032b72-c5c8-4544-9690-8e934934f336 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:13.793538 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92032b72-c5c8-4544-9690-8e934934f336 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:13.808108 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-92032b72-c5c8-4544-9690-8e934934f336 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:13.808108 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:13.808108 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:13.808108 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:13.808108 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:13.808108 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:13.808108 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:13.808108 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:13.808108 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:13.808108 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:13.808108 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:13.808108 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:13.808108 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:18:13.808108 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:18:13.808108 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:18:13.808108 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:18:13.809101 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:13.809101 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:13.809101 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:13.809101 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:13.809101 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:13.809101 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:13.809101 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:13.809101 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:13.809101 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:13.809101 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:13.809101 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:13.809101 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:13.809101 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:13.809101 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:13.809101 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:13.809101 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:13.809101 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:18:13.809977 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:18:13.809977 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:13.809977 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:18:13.809977 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 266/510] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:18:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:13.810187 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92032b72-c5c8-4544-9690-8e934934f336 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:13.821046 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:13.827599 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad96a41d-c2ed-4f17-bb38-92ff892873a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:13.828205 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad96a41d-c2ed-4f17-bb38-92ff892873a4 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:13.828482 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad96a41d-c2ed-4f17-bb38-92ff892873a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:13.828751 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad96a41d-c2ed-4f17-bb38-92ff892873a4 None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:13.829416 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad96a41d-c2ed-4f17-bb38-92ff892873a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:13.829720 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad96a41d-c2ed-4f17-bb38-92ff892873a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:13.831106 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad96a41d-c2ed-4f17-bb38-92ff892873a4 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:13.844261 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 245/511] 10.222.0.102 () {56 vars in 1114 bytes} [Thu Sep 1 17:18:13 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:13.845083 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad96a41d-c2ed-4f17-bb38-92ff892873a4 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:13.860097 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:13.918492 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92032b72-c5c8-4544-9690-8e934934f336 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:13.952585 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad96a41d-c2ed-4f17-bb38-92ff892873a4 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:13.952763 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d02735-3844-45e6-bc1c-12797cc7da5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:13.953199 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d02735-3844-45e6-bc1c-12797cc7da5f None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:13.953506 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d02735-3844-45e6-bc1c-12797cc7da5f None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:13.953779 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d02735-3844-45e6-bc1c-12797cc7da5f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:13.961046 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d02735-3844-45e6-bc1c-12797cc7da5f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:13.962783 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d02735-3844-45e6-bc1c-12797cc7da5f None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:13.974288 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-c2d02735-3844-45e6-bc1c-12797cc7da5f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 17:18:13.974288 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:13.974288 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:13.974288 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:13.974288 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:13.974288 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:13.974288 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:13.974288 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:13.974288 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:13.974288 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:13.974288 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:13.974288 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:13.974288 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:18:13.974288 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:18:13.974288 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:18:13.974288 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:18:13.975202 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:13.975202 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:13.975202 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:18:13.975202 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:18:13.975202 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:18:13.975202 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:18:13.975202 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:13.975202 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:13.975202 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:13.975202 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:13.975202 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:13.975202 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:13.975202 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:13.975202 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:13.975202 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:13.975202 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:13.975202 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:13.975202 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:13.976068 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:13.976068 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:13.976068 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:18:13.976068 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:18:13.976068 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:18:13.976068 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:18:13.976068 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:18:13.976068 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:18:13.976068 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 17:18:13.976068 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:18:13.976068 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 267/512] 10.222.0.102 () {56 vars in 1104 bytes} [Thu Sep 1 17:18:13 2022] GET /identity/v3/users/cinder => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:13.976622 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d02735-3844-45e6-bc1c-12797cc7da5f None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:13.992046 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e24939-cf5c-4361-b5ab-4122493385ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:13.992655 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e24939-cf5c-4361-b5ab-4122493385ab None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:13.992935 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e24939-cf5c-4361-b5ab-4122493385ab None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:13.993207 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e24939-cf5c-4361-b5ab-4122493385ab None admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:13.993976 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e24939-cf5c-4361-b5ab-4122493385ab None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:13.994281 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e24939-cf5c-4361-b5ab-4122493385ab None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:13.995701 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e24939-cf5c-4361-b5ab-4122493385ab None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:14.031642 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 246/513] 10.222.0.102 () {56 vars in 1142 bytes} [Thu Sep 1 17:18:13 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 374 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:14.032573 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e24939-cf5c-4361-b5ab-4122493385ab None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:14.078054 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d02735-3844-45e6-bc1c-12797cc7da5f None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:14.080673 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:14.082609 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:14.112594 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b611d4-c855-4d6f-81c7-1b7202285f9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:14.113221 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b611d4-c855-4d6f-81c7-1b7202285f9d None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:14.114126 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b611d4-c855-4d6f-81c7-1b7202285f9d None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:14.114401 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b611d4-c855-4d6f-81c7-1b7202285f9d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:14.125346 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b611d4-c855-4d6f-81c7-1b7202285f9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:14.126804 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b611d4-c855-4d6f-81c7-1b7202285f9d None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:14.134313 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e24939-cf5c-4361-b5ab-4122493385ab None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:14.141329 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-24b611d4-c855-4d6f-81c7-1b7202285f9d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:14.141329 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:14.141329 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:14.141329 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:14.141329 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:14.141329 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:14.141329 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:14.141329 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:14.141329 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:14.141329 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:14.141329 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:14.141329 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:14.141329 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:18:14.141329 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:18:14.141329 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:18:14.141329 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:18:14.142380 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:14.142380 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:14.142380 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:14.142380 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:14.142380 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:14.142380 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:14.142380 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:14.142380 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:14.142380 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:14.142380 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:14.142380 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:14.142380 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:14.142380 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:14.142380 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:14.142380 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:14.142380 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:14.142380 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:18:14.143233 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:18:14.143233 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:14.143233 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:18:14.143233 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 268/514] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:18:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:14.143233 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b611d4-c855-4d6f-81c7-1b7202285f9d None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:14.173090 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3257091c-3a71-4757-b447-66d1c19fa472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:14.174013 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3257091c-3a71-4757-b447-66d1c19fa472 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:14.174398 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3257091c-3a71-4757-b447-66d1c19fa472 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:14.174926 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3257091c-3a71-4757-b447-66d1c19fa472 None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:14.175784 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3257091c-3a71-4757-b447-66d1c19fa472 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:14.176112 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3257091c-3a71-4757-b447-66d1c19fa472 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:14.177522 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3257091c-3a71-4757-b447-66d1c19fa472 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:14.189515 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 247/515] 10.222.0.102 () {56 vars in 1114 bytes} [Thu Sep 1 17:18:14 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:14.190344 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3257091c-3a71-4757-b447-66d1c19fa472 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:14.243418 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b611d4-c855-4d6f-81c7-1b7202285f9d None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:14.277757 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca66c09-d8b6-40e7-8b92-bea3188bc931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:14.278375 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca66c09-d8b6-40e7-8b92-bea3188bc931 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:14.278655 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca66c09-d8b6-40e7-8b92-bea3188bc931 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:14.278925 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca66c09-d8b6-40e7-8b92-bea3188bc931 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:14.286222 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca66c09-d8b6-40e7-8b92-bea3188bc931 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:14.287745 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca66c09-d8b6-40e7-8b92-bea3188bc931 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:14.295322 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3257091c-3a71-4757-b447-66d1c19fa472 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:14.298761 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-3ca66c09-d8b6-40e7-8b92-bea3188bc931 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 01 17:18:14.298761 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:14.298761 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:14.298761 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:14.298761 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:14.298761 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:14.298761 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:14.298761 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:14.298761 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:14.298761 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:14.298761 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:14.298761 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:14.298761 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:18:14.298761 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:18:14.298761 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:18:14.298761 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:18:14.299811 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:14.299811 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:14.299811 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:18:14.299811 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:18:14.299811 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:18:14.299811 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:18:14.299811 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:14.299811 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:14.299811 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:14.299811 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:14.299811 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:14.299811 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:14.299811 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:14.299811 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:14.299811 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:14.299811 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:14.299811 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:14.299811 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:14.306083 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:14.306083 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:14.306083 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:18:14.306083 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:18:14.306083 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:18:14.306083 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:18:14.306083 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:18:14.306083 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:18:14.306083 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 01 17:18:14.306083 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:18:14.306083 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 269/516] 10.222.0.102 () {56 vars in 1107 bytes} [Thu Sep 1 17:18:14 2022] GET /identity/v3/users/neutron => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:14.306083 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca66c09-d8b6-40e7-8b92-bea3188bc931 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:14.306709 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:14.323585 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:14.333196 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06cd0c4-7fe5-4a48-a515-bd6add399612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:14.333781 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06cd0c4-7fe5-4a48-a515-bd6add399612 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:14.334062 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06cd0c4-7fe5-4a48-a515-bd6add399612 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:14.334339 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06cd0c4-7fe5-4a48-a515-bd6add399612 None admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:14.335069 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06cd0c4-7fe5-4a48-a515-bd6add399612 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:14.335380 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06cd0c4-7fe5-4a48-a515-bd6add399612 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:14.336838 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06cd0c4-7fe5-4a48-a515-bd6add399612 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:14.355601 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 248/517] 10.222.0.102 () {56 vars in 1142 bytes} [Thu Sep 1 17:18:14 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 374 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:14.356772 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06cd0c4-7fe5-4a48-a515-bd6add399612 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:14.406813 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca66c09-d8b6-40e7-8b92-bea3188bc931 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:14.439879 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f40399-7bdf-413d-8f32-61e01ce11858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:14.440485 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f40399-7bdf-413d-8f32-61e01ce11858 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:14.440762 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f40399-7bdf-413d-8f32-61e01ce11858 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:14.441041 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f40399-7bdf-413d-8f32-61e01ce11858 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:14.455913 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06cd0c4-7fe5-4a48-a515-bd6add399612 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:14.490244 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d55c544-3705-4257-8c44-83c4fedf85fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:14.490843 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d55c544-3705-4257-8c44-83c4fedf85fe None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:14.491124 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d55c544-3705-4257-8c44-83c4fedf85fe None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:14.491398 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d55c544-3705-4257-8c44-83c4fedf85fe None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:14.492231 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d55c544-3705-4257-8c44-83c4fedf85fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:14.492358 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d55c544-3705-4257-8c44-83c4fedf85fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:14.493746 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d55c544-3705-4257-8c44-83c4fedf85fe None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:14.505753 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 249/518] 10.222.0.102 () {56 vars in 1114 bytes} [Thu Sep 1 17:18:14 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:14.506557 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d55c544-3705-4257-8c44-83c4fedf85fe None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:14.507473 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f40399-7bdf-413d-8f32-61e01ce11858 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:14.508944 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f40399-7bdf-413d-8f32-61e01ce11858 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:14.525844 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:14.536639 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:14.609418 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 270/519] 10.222.0.102 () {58 vars in 1287 bytes} [Thu Sep 1 17:18:14 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 17:18:14.609418 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f40399-7bdf-413d-8f32-61e01ce11858 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:14.616083 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d55c544-3705-4257-8c44-83c4fedf85fe None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:14.651474 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1de08cc-0e02-42b3-8c0d-d9bb74981ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:14.652101 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1de08cc-0e02-42b3-8c0d-d9bb74981ddf None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:14.652385 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1de08cc-0e02-42b3-8c0d-d9bb74981ddf None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:14.652667 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1de08cc-0e02-42b3-8c0d-d9bb74981ddf None admin] PATH_INFO: `/v3/projects/service` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:14.659174 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1de08cc-0e02-42b3-8c0d-d9bb74981ddf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:14.661148 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1de08cc-0e02-42b3-8c0d-d9bb74981ddf None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:14.672905 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-c1de08cc-0e02-42b3-8c0d-d9bb74981ddf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:18:14.672905 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:14.672905 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:14.672905 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:14.672905 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:14.672905 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:14.672905 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:14.672905 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:14.672905 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:14.672905 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:14.672905 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:14.672905 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:14.672905 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:18:14.672905 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:18:14.672905 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:18:14.673762 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:18:14.673762 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:14.673762 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:14.673762 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:14.673762 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:14.673762 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:14.673762 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:14.673762 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:14.673762 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:14.673762 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:14.673762 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:14.673762 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:14.673762 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:14.673762 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:14.673762 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:14.673762 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:14.673762 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:14.674585 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:18:14.674585 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:18:14.674585 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:18:14.674585 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:18:14.674585 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:18:14.674585 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:18:14.674585 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:18:14.674585 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:18:14.674585 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 250/520] 10.222.0.102 () {56 vars in 1116 bytes} [Thu Sep 1 17:18:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:14.675067 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1de08cc-0e02-42b3-8c0d-d9bb74981ddf None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:14.706395 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f40399-7bdf-413d-8f32-61e01ce11858 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:14.722219 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:14.741438 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426b7bd3-8024-4e18-b026-49ae3182b0b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:14.742036 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426b7bd3-8024-4e18-b026-49ae3182b0b5 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:14.742313 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426b7bd3-8024-4e18-b026-49ae3182b0b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:14.742585 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426b7bd3-8024-4e18-b026-49ae3182b0b5 None admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:14.743297 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426b7bd3-8024-4e18-b026-49ae3182b0b5 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:14.743598 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426b7bd3-8024-4e18-b026-49ae3182b0b5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:14.745026 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426b7bd3-8024-4e18-b026-49ae3182b0b5 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:14.761476 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 271/521] 10.222.0.102 () {56 vars in 1144 bytes} [Thu Sep 1 17:18:14 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 376 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:14.762253 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426b7bd3-8024-4e18-b026-49ae3182b0b5 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:14.767519 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:14.782633 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1de08cc-0e02-42b3-8c0d-d9bb74981ddf None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:14.818435 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e45cc4-6d8c-44d4-9bf3-ae11749cba52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:14.819054 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e45cc4-6d8c-44d4-9bf3-ae11749cba52 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:14.819343 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e45cc4-6d8c-44d4-9bf3-ae11749cba52 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:14.819617 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e45cc4-6d8c-44d4-9bf3-ae11749cba52 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:14.860712 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426b7bd3-8024-4e18-b026-49ae3182b0b5 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:14.888793 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e45cc4-6d8c-44d4-9bf3-ae11749cba52 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:14.890292 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e45cc4-6d8c-44d4-9bf3-ae11749cba52 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:14.895818 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4eef78a-e926-40e8-9e29-5eebbc8db8c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:14.896435 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eef78a-e926-40e8-9e29-5eebbc8db8c1 None admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:14.896700 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eef78a-e926-40e8-9e29-5eebbc8db8c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:14.896970 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eef78a-e926-40e8-9e29-5eebbc8db8c1 None admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:14.897654 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4eef78a-e926-40e8-9e29-5eebbc8db8c1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:14.899489 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4eef78a-e926-40e8-9e29-5eebbc8db8c1 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:14.920995 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 272/522] 10.222.0.102 () {60 vars in 1137 bytes} [Thu Sep 1 17:18:14 2022] POST /identity/v3/users => generated 248 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:18:14.921970 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4eef78a-e926-40e8-9e29-5eebbc8db8c1 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:14.957408 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:14.991222 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:15.010675 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 251/523] 10.222.0.102 () {58 vars in 1287 bytes} [Thu Sep 1 17:18:14 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 17:18:15.011667 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e45cc4-6d8c-44d4-9bf3-ae11749cba52 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:15.045990 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4eef78a-e926-40e8-9e29-5eebbc8db8c1 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:15.090977 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3497916-b991-4572-802d-1e0639c69768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:15.091611 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3497916-b991-4572-802d-1e0639c69768 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:15.091888 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3497916-b991-4572-802d-1e0639c69768 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:15.093090 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3497916-b991-4572-802d-1e0639c69768 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:15.104899 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3497916-b991-4572-802d-1e0639c69768 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:15.106539 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3497916-b991-4572-802d-1e0639c69768 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:15.122364 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-b3497916-b991-4572-802d-1e0639c69768 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:15.122364 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:15.122364 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:15.122364 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:15.122364 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:15.122364 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:15.122364 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:15.122364 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:15.122364 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:15.122364 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:15.122364 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:15.122364 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:15.122364 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:18:15.122364 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:18:15.122364 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:18:15.122364 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:18:15.123562 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:15.123562 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:15.123562 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:15.123562 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:15.123562 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:15.123562 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:15.123562 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:15.123562 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:15.123562 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:15.123562 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:15.123562 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:15.123562 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:15.123562 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:15.123562 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:15.123562 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:15.123562 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:15.123562 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:18:15.124578 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:18:15.124578 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:15.124578 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:18:15.124578 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 273/524] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:18:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:15.124578 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3497916-b991-4572-802d-1e0639c69768 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:15.136537 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e45cc4-6d8c-44d4-9bf3-ae11749cba52 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:15.170471 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8e8ce4-d451-4088-9480-f40d15bec540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:15.171212 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8e8ce4-d451-4088-9480-f40d15bec540 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:15.171499 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8e8ce4-d451-4088-9480-f40d15bec540 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:15.171743 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8e8ce4-d451-4088-9480-f40d15bec540 None admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:15.172436 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8e8ce4-d451-4088-9480-f40d15bec540 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:15.172787 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8e8ce4-d451-4088-9480-f40d15bec540 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:15.174291 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8e8ce4-d451-4088-9480-f40d15bec540 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:15.197471 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 252/525] 10.222.0.102 () {56 vars in 1153 bytes} [Thu Sep 1 17:18:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:15.197471 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8e8ce4-d451-4088-9480-f40d15bec540 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:15.221350 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:15.242638 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3497916-b991-4572-802d-1e0639c69768 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:15.243247 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:15.279010 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb7b431-ef31-4733-927c-3cfa3333c929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:15.279616 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb7b431-ef31-4733-927c-3cfa3333c929 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:15.279891 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb7b431-ef31-4733-927c-3cfa3333c929 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:15.280168 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb7b431-ef31-4733-927c-3cfa3333c929 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:15.300157 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8e8ce4-d451-4088-9480-f40d15bec540 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:15.335703 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0e6b49-7516-43ba-a4f2-43e9ddd25263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:15.336338 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0e6b49-7516-43ba-a4f2-43e9ddd25263 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:15.336620 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0e6b49-7516-43ba-a4f2-43e9ddd25263 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:15.336948 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0e6b49-7516-43ba-a4f2-43e9ddd25263 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:15.350235 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0e6b49-7516-43ba-a4f2-43e9ddd25263 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:15.351802 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb7b431-ef31-4733-927c-3cfa3333c929 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:15.352202 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0e6b49-7516-43ba-a4f2-43e9ddd25263 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:15.353540 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb7b431-ef31-4733-927c-3cfa3333c929 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:15.367763 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-9d0e6b49-7516-43ba-a4f2-43e9ddd25263 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:15.367763 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:15.367763 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:15.367763 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:15.367763 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:15.367763 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:15.367763 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:15.367763 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:15.367763 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:15.367763 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:15.367763 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:15.367763 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:15.367763 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:18:15.367763 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:18:15.367763 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:18:15.367763 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:18:15.368753 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:15.368753 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:15.368753 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:15.368753 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:15.368753 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:15.368753 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:15.368753 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:15.368753 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:15.368753 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:15.368753 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:15.368753 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:15.368753 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:15.368753 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:15.368753 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:15.368753 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:15.368753 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:15.368753 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:18:15.369725 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:18:15.369725 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:15.369725 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:18:15.370472 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 253/526] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:18:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:15.371324 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0e6b49-7516-43ba-a4f2-43e9ddd25263 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:15.452887 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:15.472343 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 274/527] 10.222.0.102 () {58 vars in 1287 bytes} [Thu Sep 1 17:18:15 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 17:18:15.478705 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb7b431-ef31-4733-927c-3cfa3333c929 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:15.486672 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:15.492299 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0e6b49-7516-43ba-a4f2-43e9ddd25263 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:15.530983 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307b5044-9770-404c-b6b0-6a06997f4145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:15.531596 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307b5044-9770-404c-b6b0-6a06997f4145 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:15.531881 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307b5044-9770-404c-b6b0-6a06997f4145 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:15.532157 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307b5044-9770-404c-b6b0-6a06997f4145 None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:15.532799 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307b5044-9770-404c-b6b0-6a06997f4145 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:15.533107 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307b5044-9770-404c-b6b0-6a06997f4145 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:15.534523 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307b5044-9770-404c-b6b0-6a06997f4145 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:15.552252 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 254/528] 10.222.0.102 () {56 vars in 1114 bytes} [Thu Sep 1 17:18:15 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:15.552252 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307b5044-9770-404c-b6b0-6a06997f4145 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:15.591605 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb7b431-ef31-4733-927c-3cfa3333c929 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:15.626494 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ce97fe-62d5-4bf3-800b-ad92b5ce422e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:15.627092 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ce97fe-62d5-4bf3-800b-ad92b5ce422e None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:15.627381 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ce97fe-62d5-4bf3-800b-ad92b5ce422e None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:15.627654 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ce97fe-62d5-4bf3-800b-ad92b5ce422e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:15.628375 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ce97fe-62d5-4bf3-800b-ad92b5ce422e None admin] RBAC: Adding query filter params (role.id=f21e23654e4740158c7d4f46e726e515, scope.project.id=e78fff7f1f6e4f68b8d233ad2e5bbd4f, user.id=5f8293316baf4774b62fc11d55d50865) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:15.628685 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ce97fe-62d5-4bf3-800b-ad92b5ce422e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:15.630256 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ce97fe-62d5-4bf3-800b-ad92b5ce422e None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:15.636856 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 275/529] 10.222.0.102 () {56 vars in 1379 bytes} [Thu Sep 1 17:18:15 2022] GET /identity/v3/role_assignments?user.id=5f8293316baf4774b62fc11d55d50865&scope.project.id=e78fff7f1f6e4f68b8d233ad2e5bbd4f&role.id=f21e23654e4740158c7d4f46e726e515 => generated 260 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:15.642512 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ce97fe-62d5-4bf3-800b-ad92b5ce422e None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:15.669706 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307b5044-9770-404c-b6b0-6a06997f4145 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:15.706153 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:15.727182 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d1dd84-4055-46b4-8463-c96d6def5dd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:15.728204 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d1dd84-4055-46b4-8463-c96d6def5dd7 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:15.728621 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d1dd84-4055-46b4-8463-c96d6def5dd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:15.728905 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d1dd84-4055-46b4-8463-c96d6def5dd7 None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:15.729596 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d1dd84-4055-46b4-8463-c96d6def5dd7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:15.729906 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d1dd84-4055-46b4-8463-c96d6def5dd7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:15.731313 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d1dd84-4055-46b4-8463-c96d6def5dd7 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:15.748521 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 255/530] 10.222.0.102 () {56 vars in 1114 bytes} [Thu Sep 1 17:18:15 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:15.749913 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d1dd84-4055-46b4-8463-c96d6def5dd7 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:15.774473 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ce97fe-62d5-4bf3-800b-ad92b5ce422e None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:15.778674 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:15.826012 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeba17a5-b81c-4cfb-afef-d6eb64a5ce0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:15.827763 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeba17a5-b81c-4cfb-afef-d6eb64a5ce0c None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:15.828057 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeba17a5-b81c-4cfb-afef-d6eb64a5ce0c None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:15.828335 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeba17a5-b81c-4cfb-afef-d6eb64a5ce0c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:15.845447 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeba17a5-b81c-4cfb-afef-d6eb64a5ce0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:15.846910 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeba17a5-b81c-4cfb-afef-d6eb64a5ce0c None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:15.864634 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-eeba17a5-b81c-4cfb-afef-d6eb64a5ce0c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:15.864634 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:15.864634 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:15.864634 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:15.864634 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:15.864634 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:15.864634 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:15.864634 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:15.864634 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:15.864634 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:15.864634 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:15.864634 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:15.864634 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:18:15.864634 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:18:15.864634 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:18:15.864634 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:18:15.866024 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:15.866024 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:15.866024 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:15.866024 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:15.866024 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:15.866024 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:15.866024 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:15.866024 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:15.866024 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:15.866024 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:15.866024 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:15.866024 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:15.866024 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:15.866024 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:15.866024 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:15.866024 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:15.866024 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:18:15.866868 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:18:15.866868 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:15.866868 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:18:15.866868 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 276/531] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:18:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:15.867546 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeba17a5-b81c-4cfb-afef-d6eb64a5ce0c None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:15.882564 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d1dd84-4055-46b4-8463-c96d6def5dd7 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:15.918474 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118ae9a8-31d7-497d-b920-ab84e2e3f7a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:15.919350 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118ae9a8-31d7-497d-b920-ab84e2e3f7a2 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:15.919732 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118ae9a8-31d7-497d-b920-ab84e2e3f7a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:15.920103 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118ae9a8-31d7-497d-b920-ab84e2e3f7a2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:15.927554 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118ae9a8-31d7-497d-b920-ab84e2e3f7a2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:15.929130 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118ae9a8-31d7-497d-b920-ab84e2e3f7a2 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:15.941050 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-118ae9a8-31d7-497d-b920-ab84e2e3f7a2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:18:15.941050 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:15.941050 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:15.941050 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:15.941050 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:15.941050 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:15.941050 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:15.941050 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:15.941050 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:15.941050 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:15.941050 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:15.941050 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:15.941050 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:18:15.941050 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:18:15.941050 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:18:15.942964 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:18:15.942964 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:15.942964 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:15.942964 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:15.942964 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:15.942964 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:15.942964 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:15.942964 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:15.942964 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:15.942964 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:15.942964 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:15.942964 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:15.942964 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:15.942964 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:15.942964 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:15.942964 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:15.942964 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:15.945027 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:18:15.945027 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:18:15.945027 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:18:15.945027 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:18:15.945027 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:18:15.945027 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:18:15.945027 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:18:15.945027 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:18:15.953348 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 256/532] 10.222.0.102 () {56 vars in 1116 bytes} [Thu Sep 1 17:18:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:15.953348 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118ae9a8-31d7-497d-b920-ab84e2e3f7a2 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:15.981095 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeba17a5-b81c-4cfb-afef-d6eb64a5ce0c None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:15.997757 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:16.022001 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce8d8e5-fe70-428d-868f-9c70a2ce39c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:16.022694 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce8d8e5-fe70-428d-868f-9c70a2ce39c0 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:16.022964 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce8d8e5-fe70-428d-868f-9c70a2ce39c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:16.023242 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce8d8e5-fe70-428d-868f-9c70a2ce39c0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:16.030908 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce8d8e5-fe70-428d-868f-9c70a2ce39c0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:16.031621 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:16.033285 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce8d8e5-fe70-428d-868f-9c70a2ce39c0 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:16.048463 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-5ce8d8e5-fe70-428d-868f-9c70a2ce39c0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:18:16.048463 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:16.048463 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:16.048463 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:16.048463 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:16.048463 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:16.048463 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:16.048463 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:16.048463 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:16.048463 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:16.048463 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:16.048463 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:16.048463 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:18:16.048463 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:18:16.048463 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:18:16.050423 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:18:16.050423 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:16.050423 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:16.050423 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:16.050423 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:16.050423 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:16.050423 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:16.050423 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:16.050423 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:16.050423 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:16.050423 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:16.050423 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:16.050423 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:16.050423 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:16.050423 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:16.050423 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:16.050423 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:16.052542 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:18:16.052542 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:18:16.052542 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:18:16.052542 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:18:16.052542 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:18:16.052542 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:18:16.052542 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:18:16.052542 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:18:16.061466 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 277/533] 10.222.0.102 () {56 vars in 1116 bytes} [Thu Sep 1 17:18:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:16.061466 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce8d8e5-fe70-428d-868f-9c70a2ce39c0 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:16.063029 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118ae9a8-31d7-497d-b920-ab84e2e3f7a2 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:16.094713 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db11174-143c-4d90-b158-6edf7fc2b51e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:16.095323 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db11174-143c-4d90-b158-6edf7fc2b51e None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:16.095604 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db11174-143c-4d90-b158-6edf7fc2b51e None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:16.095945 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db11174-143c-4d90-b158-6edf7fc2b51e None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:16.096592 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db11174-143c-4d90-b158-6edf7fc2b51e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:16.096896 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db11174-143c-4d90-b158-6edf7fc2b51e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:16.098895 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db11174-143c-4d90-b158-6edf7fc2b51e None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:16.112420 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 257/534] 10.222.0.102 () {56 vars in 1114 bytes} [Thu Sep 1 17:18:15 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:16.113414 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db11174-143c-4d90-b158-6edf7fc2b51e None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:16.177403 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce8d8e5-fe70-428d-868f-9c70a2ce39c0 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:16.214971 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75f40d2-9d9d-42a0-a920-921a4404f94c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:16.215584 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75f40d2-9d9d-42a0-a920-921a4404f94c None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:16.215986 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75f40d2-9d9d-42a0-a920-921a4404f94c None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:16.216325 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75f40d2-9d9d-42a0-a920-921a4404f94c None admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:16.217073 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75f40d2-9d9d-42a0-a920-921a4404f94c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:16.217415 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75f40d2-9d9d-42a0-a920-921a4404f94c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:16.218847 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75f40d2-9d9d-42a0-a920-921a4404f94c None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:16.229661 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db11174-143c-4d90-b158-6edf7fc2b51e None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:16.243694 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 278/535] 10.222.0.102 () {56 vars in 1153 bytes} [Thu Sep 1 17:18:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:16.243694 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75f40d2-9d9d-42a0-a920-921a4404f94c None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:16.263305 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:16.286528 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fecf567-e704-4b62-8849-7eab9e7ab68a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:16.287983 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fecf567-e704-4b62-8849-7eab9e7ab68a None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:16.288443 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fecf567-e704-4b62-8849-7eab9e7ab68a None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:16.288897 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fecf567-e704-4b62-8849-7eab9e7ab68a None admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:16.290229 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fecf567-e704-4b62-8849-7eab9e7ab68a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:16.291157 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fecf567-e704-4b62-8849-7eab9e7ab68a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:16.293096 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fecf567-e704-4b62-8849-7eab9e7ab68a None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:16.301419 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:16.313595 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 258/536] 10.222.0.102 () {56 vars in 1153 bytes} [Thu Sep 1 17:18:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:16.314641 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fecf567-e704-4b62-8849-7eab9e7ab68a None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:16.366029 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75f40d2-9d9d-42a0-a920-921a4404f94c None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:16.403150 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89da6f32-d4f4-4f11-9030-1815f65aebca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:16.403974 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89da6f32-d4f4-4f11-9030-1815f65aebca None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:16.404420 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89da6f32-d4f4-4f11-9030-1815f65aebca None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:16.404797 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89da6f32-d4f4-4f11-9030-1815f65aebca None admin] PATH_INFO: `/v3/projects/service` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:16.412657 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89da6f32-d4f4-4f11-9030-1815f65aebca None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:16.414262 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89da6f32-d4f4-4f11-9030-1815f65aebca None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:16.424071 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fecf567-e704-4b62-8849-7eab9e7ab68a None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:16.428193 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-89da6f32-d4f4-4f11-9030-1815f65aebca None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:18:16.428193 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:16.428193 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:16.428193 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:16.428193 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:16.428193 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:16.428193 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:16.428193 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:16.428193 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:16.428193 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:16.428193 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:16.428193 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:16.428193 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:18:16.428193 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:18:16.428193 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:18:16.429043 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:18:16.429043 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:16.429043 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:16.429043 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:16.429043 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:16.429043 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:16.429043 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:16.429043 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:16.429043 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:16.429043 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:16.429043 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:16.429043 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:16.429043 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:16.429043 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:16.429043 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:16.429043 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:16.429043 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:16.429998 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:18:16.429998 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:18:16.429998 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:18:16.429998 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:18:16.429998 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:18:16.429998 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:18:16.429998 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:18:16.429998 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:18:16.430920 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 279/537] 10.222.0.102 () {56 vars in 1116 bytes} [Thu Sep 1 17:18:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:16.431722 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89da6f32-d4f4-4f11-9030-1815f65aebca None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:16.476615 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3042d9-63a6-4d3d-ba1a-876a73813de7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:16.477556 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3042d9-63a6-4d3d-ba1a-876a73813de7 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:16.478117 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3042d9-63a6-4d3d-ba1a-876a73813de7 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:16.479244 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3042d9-63a6-4d3d-ba1a-876a73813de7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:16.480090 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3042d9-63a6-4d3d-ba1a-876a73813de7 None admin] RBAC: Adding query filter params (role.id=f21e23654e4740158c7d4f46e726e515, scope.project.id=e78fff7f1f6e4f68b8d233ad2e5bbd4f, user.id=039683b8d1d146c6add5ffa235d06f91) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:16.480439 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3042d9-63a6-4d3d-ba1a-876a73813de7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:16.481914 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3042d9-63a6-4d3d-ba1a-876a73813de7 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:16.493431 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 259/538] 10.222.0.102 () {56 vars in 1379 bytes} [Thu Sep 1 17:18:16 2022] GET /identity/v3/role_assignments?user.id=039683b8d1d146c6add5ffa235d06f91&scope.project.id=e78fff7f1f6e4f68b8d233ad2e5bbd4f&role.id=f21e23654e4740158c7d4f46e726e515 => generated 260 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:16.494492 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3042d9-63a6-4d3d-ba1a-876a73813de7 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:16.573169 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:16.621859 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89da6f32-d4f4-4f11-9030-1815f65aebca None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:16.641681 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:16.674070 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a169043-2a8d-45b4-918b-0dd1e09bbcd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:16.674682 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a169043-2a8d-45b4-918b-0dd1e09bbcd2 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:16.674962 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a169043-2a8d-45b4-918b-0dd1e09bbcd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:16.675233 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a169043-2a8d-45b4-918b-0dd1e09bbcd2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:16.676014 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a169043-2a8d-45b4-918b-0dd1e09bbcd2 None admin] RBAC: Adding query filter params (role.id=f21e23654e4740158c7d4f46e726e515, scope.project.id=e78fff7f1f6e4f68b8d233ad2e5bbd4f, user.id=523ed589872a45a09b6a7675a7fb0b75) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:16.676326 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a169043-2a8d-45b4-918b-0dd1e09bbcd2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:16.677800 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a169043-2a8d-45b4-918b-0dd1e09bbcd2 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:16.684900 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 280/539] 10.222.0.102 () {56 vars in 1379 bytes} [Thu Sep 1 17:18:16 2022] GET /identity/v3/role_assignments?user.id=523ed589872a45a09b6a7675a7fb0b75&scope.project.id=e78fff7f1f6e4f68b8d233ad2e5bbd4f&role.id=f21e23654e4740158c7d4f46e726e515 => generated 260 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:16.699488 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3042d9-63a6-4d3d-ba1a-876a73813de7 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:16.738773 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437fa9e4-c2fa-49c1-9ad0-21552564b36b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:16.739389 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437fa9e4-c2fa-49c1-9ad0-21552564b36b None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:16.739669 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437fa9e4-c2fa-49c1-9ad0-21552564b36b None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:16.739945 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437fa9e4-c2fa-49c1-9ad0-21552564b36b None admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:16.740642 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437fa9e4-c2fa-49c1-9ad0-21552564b36b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:16.740953 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437fa9e4-c2fa-49c1-9ad0-21552564b36b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:16.742435 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437fa9e4-c2fa-49c1-9ad0-21552564b36b None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:16.757040 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 260/540] 10.222.0.102 () {56 vars in 1153 bytes} [Thu Sep 1 17:18:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:16.762992 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a169043-2a8d-45b4-918b-0dd1e09bbcd2 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:16.882294 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a169043-2a8d-45b4-918b-0dd1e09bbcd2 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:16.934359 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b584908-4fd5-4480-8f26-8b6e3264ca78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:16.934979 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b584908-4fd5-4480-8f26-8b6e3264ca78 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:16.935261 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b584908-4fd5-4480-8f26-8b6e3264ca78 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:16.935535 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b584908-4fd5-4480-8f26-8b6e3264ca78 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:16.936264 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b584908-4fd5-4480-8f26-8b6e3264ca78 None admin] RBAC: Adding query filter params (role.id=f21e23654e4740158c7d4f46e726e515, scope.project.id=e78fff7f1f6e4f68b8d233ad2e5bbd4f, user.id=7c7d3594fdab4191be1d841af43a5c79) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:16.936573 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b584908-4fd5-4480-8f26-8b6e3264ca78 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:16.938026 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b584908-4fd5-4480-8f26-8b6e3264ca78 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:16.944561 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 281/541] 10.222.0.102 () {56 vars in 1379 bytes} [Thu Sep 1 17:18:16 2022] GET /identity/v3/role_assignments?user.id=7c7d3594fdab4191be1d841af43a5c79&scope.project.id=e78fff7f1f6e4f68b8d233ad2e5bbd4f&role.id=f21e23654e4740158c7d4f46e726e515 => generated 260 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:17.048537 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70849f6f-83d4-4e25-b276-cf5ce9cd242c None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:17.049065 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70849f6f-83d4-4e25-b276-cf5ce9cd242c None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:17.049065 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70849f6f-83d4-4e25-b276-cf5ce9cd242c None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:17.050599 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 261/542] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:18:17 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:18:17.056460 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b92d0b-b409-4798-9c18-b22b90292a37 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:17.057067 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b92d0b-b409-4798-9c18-b22b90292a37 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:17.057611 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b92d0b-b409-4798-9c18-b22b90292a37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:17.083086 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:17.089524 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-68b92d0b-b409-4798-9c18-b22b90292a37 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:18:17.091511 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-68b92d0b-b409-4798-9c18-b22b90292a37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:18:17.163924 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 282/543] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:18:17 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:18:17.480279 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70849f6f-83d4-4e25-b276-cf5ce9cd242c None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:17.611446 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70849f6f-83d4-4e25-b276-cf5ce9cd242c None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:17.643300 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:17.662177 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ed7563-c826-4932-8a38-a2302eb0eeab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:17.662818 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ed7563-c826-4932-8a38-a2302eb0eeab None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:17.663094 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ed7563-c826-4932-8a38-a2302eb0eeab None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:17.663361 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ed7563-c826-4932-8a38-a2302eb0eeab None admin] PATH_INFO: `/v3/roles/service` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:17.684406 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ed7563-c826-4932-8a38-a2302eb0eeab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:18:17.689286 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-28ed7563-c826-4932-8a38-a2302eb0eeab None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:18:17.689286 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:17.689286 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:17.689286 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:17.689286 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:17.689286 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:17.689286 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:17.689286 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:17.689286 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:17.689286 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:17.689286 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:17.689286 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:17.689286 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:18:17.689286 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:18:17.689286 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:18:17.689286 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:18:17.690260 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:18:17.690260 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:18:17.690260 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:18:17.690260 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:18:17.690260 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:17.690260 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:17.690260 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:17.690260 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:17.690260 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:17.690260 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:17.690260 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:17.690260 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:17.690260 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:17.690260 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:17.690260 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:17.690260 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:17.691029 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:17.691029 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:17.691029 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:17.691029 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:17.691029 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:18:17.691029 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:18:17.691029 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:18:17.691029 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:18:17.691029 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:18:17.691029 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:18:17.691029 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:18:17.691029 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:18:17.691029 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 262/544] 10.222.0.102 () {56 vars in 1107 bytes} [Thu Sep 1 17:18:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:17.696695 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b92d0b-b409-4798-9c18-b22b90292a37 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:17.790770 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4d4a38-1df7-450d-869c-c0f4db1d83bf None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:17.791056 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4d4a38-1df7-450d-869c-c0f4db1d83bf None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:17.791246 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4d4a38-1df7-450d-869c-c0f4db1d83bf None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:17.793971 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 263/545] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:18:17 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Sep 01 17:18:17.799698 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c288e5e0-2f90-42b9-96ec-5cb2332daf56 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:17.799988 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c288e5e0-2f90-42b9-96ec-5cb2332daf56 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:17.800262 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c288e5e0-2f90-42b9-96ec-5cb2332daf56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:17.825881 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-c288e5e0-2f90-42b9-96ec-5cb2332daf56 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:18:17.829540 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-c288e5e0-2f90-42b9-96ec-5cb2332daf56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:18:17.908745 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b92d0b-b409-4798-9c18-b22b90292a37 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:17.973439 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 264/546] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:18:17 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:18:17.975489 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06790b1e-1d2c-49ad-bb79-55df5415aaa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:17.976105 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06790b1e-1d2c-49ad-bb79-55df5415aaa9 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:17.976385 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06790b1e-1d2c-49ad-bb79-55df5415aaa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:17.976654 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06790b1e-1d2c-49ad-bb79-55df5415aaa9 None admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:17.977332 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06790b1e-1d2c-49ad-bb79-55df5415aaa9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:17.977644 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06790b1e-1d2c-49ad-bb79-55df5415aaa9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:17.979265 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06790b1e-1d2c-49ad-bb79-55df5415aaa9 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:17.989688 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:17.992456 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 283/547] 10.222.0.102 () {56 vars in 1108 bytes} [Thu Sep 1 17:18:17 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:17.993902 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c288e5e0-2f90-42b9-96ec-5cb2332daf56 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:18.117813 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c288e5e0-2f90-42b9-96ec-5cb2332daf56 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:18.137379 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:18.164383 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6686d67-bfae-4172-8634-eafed28cb233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:18.165017 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6686d67-bfae-4172-8634-eafed28cb233 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:18.165377 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6686d67-bfae-4172-8634-eafed28cb233 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:18.165656 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6686d67-bfae-4172-8634-eafed28cb233 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:18.179119 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6686d67-bfae-4172-8634-eafed28cb233 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:18.180581 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6686d67-bfae-4172-8634-eafed28cb233 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:18.199617 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-f6686d67-bfae-4172-8634-eafed28cb233 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:18.199617 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:18.199617 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:18.199617 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:18.199617 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:18.199617 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:18.199617 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:18.199617 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:18.199617 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:18.199617 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:18.199617 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:18.199617 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:18.199617 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:18:18.199617 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:18:18.199617 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:18:18.199617 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:18:18.200578 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:18.200578 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:18.200578 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:18.200578 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:18.200578 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:18.200578 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:18.200578 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:18.200578 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:18.200578 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:18.200578 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:18.200578 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:18.200578 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:18.200578 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:18.200578 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:18.200578 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:18.200578 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:18.200578 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:18:18.201417 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:18:18.201417 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:18.201417 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:18:18.205347 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 265/548] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:18:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:18.207055 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06790b1e-1d2c-49ad-bb79-55df5415aaa9 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:18.323652 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06790b1e-1d2c-49ad-bb79-55df5415aaa9 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:18.360173 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8d3bba-3496-4840-b327-a6af066f546e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:18.360793 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8d3bba-3496-4840-b327-a6af066f546e None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:18.361074 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8d3bba-3496-4840-b327-a6af066f546e None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:18.361378 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8d3bba-3496-4840-b327-a6af066f546e None admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:18.362160 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8d3bba-3496-4840-b327-a6af066f546e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:18.362462 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8d3bba-3496-4840-b327-a6af066f546e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:18.364394 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8d3bba-3496-4840-b327-a6af066f546e None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:18.381591 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 284/549] 10.222.0.102 () {56 vars in 1114 bytes} [Thu Sep 1 17:18:18 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:18.383012 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6686d67-bfae-4172-8634-eafed28cb233 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:18.394172 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8d3bba-3496-4840-b327-a6af066f546e None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:18.503430 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:18.574200 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6686d67-bfae-4172-8634-eafed28cb233 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:18.584903 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8d3bba-3496-4840-b327-a6af066f546e None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:18.639320 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf53bed-76b5-4fa2-b6f9-34e79b282e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:18.639937 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf53bed-76b5-4fa2-b6f9-34e79b282e06 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:18.640218 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf53bed-76b5-4fa2-b6f9-34e79b282e06 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:18.640492 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf53bed-76b5-4fa2-b6f9-34e79b282e06 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:18.648476 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf53bed-76b5-4fa2-b6f9-34e79b282e06 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:18.648912 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2401f7-3f17-4d5e-a338-c8e817258621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:18.649602 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2401f7-3f17-4d5e-a338-c8e817258621 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:18.649883 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2401f7-3f17-4d5e-a338-c8e817258621 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:18.650062 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf53bed-76b5-4fa2-b6f9-34e79b282e06 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:18.650219 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2401f7-3f17-4d5e-a338-c8e817258621 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:18.662284 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-aaf53bed-76b5-4fa2-b6f9-34e79b282e06 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 01 17:18:18.662284 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:18.662284 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:18.662284 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:18.662284 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:18.662284 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:18.662284 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:18.662284 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:18.662284 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:18.662284 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:18.662284 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:18.662284 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:18.662284 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:18:18.662284 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:18:18.662284 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:18:18.662284 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:18:18.664946 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:18.664946 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:18.664946 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:18:18.664946 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:18:18.664946 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:18:18.664946 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:18:18.664946 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:18.664946 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:18.664946 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:18.664946 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:18.664946 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:18.664946 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:18.664946 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:18.664946 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:18.664946 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:18.664946 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:18.664946 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:18.664946 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:18.667204 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:18.667204 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:18.667204 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:18:18.667204 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:18:18.667204 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:18:18.667204 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:18:18.667204 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:18:18.667204 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:18:18.667204 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 01 17:18:18.667204 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:18:18.668731 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2401f7-3f17-4d5e-a338-c8e817258621 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:18:18.668874 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 285/550] 10.222.0.102 () {56 vars in 1101 bytes} [Thu Sep 1 17:18:18 2022] GET /identity/v3/users/swift => generated 83 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:18:18.670972 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf53bed-76b5-4fa2-b6f9-34e79b282e06 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:18.687485 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-fc2401f7-3f17-4d5e-a338-c8e817258621 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:18:18.687485 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:18.687485 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:18.687485 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:18.687485 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:18.687485 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:18.687485 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:18.687485 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:18.687485 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:18.687485 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:18.687485 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:18.687485 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:18.687485 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:18:18.687485 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:18:18.687485 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:18:18.687485 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:18:18.689639 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:18:18.689639 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:18:18.689639 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:18:18.689639 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:18:18.689639 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:18.689639 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:18.689639 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:18.689639 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:18.689639 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:18.689639 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:18.689639 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:18.689639 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:18.689639 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:18.689639 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:18.689639 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:18.689639 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:18.691719 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:18.691719 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:18.691719 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:18.691719 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:18.691719 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:18:18.691719 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:18:18.691719 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:18:18.691719 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:18:18.691719 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:18:18.691719 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:18:18.691719 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:18:18.691719 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:18:18.691719 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 266/551] 10.222.0.102 () {56 vars in 1107 bytes} [Thu Sep 1 17:18:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:18.698461 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2401f7-3f17-4d5e-a338-c8e817258621 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:18.747698 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:18.847252 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf53bed-76b5-4fa2-b6f9-34e79b282e06 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:18.861868 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2401f7-3f17-4d5e-a338-c8e817258621 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:18.866296 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:18.884818 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d24185d-e77a-4567-929f-bc16f62c7bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:18.885454 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d24185d-e77a-4567-929f-bc16f62c7bcf None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:18.885736 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d24185d-e77a-4567-929f-bc16f62c7bcf None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:18.886008 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d24185d-e77a-4567-929f-bc16f62c7bcf None admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:18.886738 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d24185d-e77a-4567-929f-bc16f62c7bcf None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:18.887043 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d24185d-e77a-4567-929f-bc16f62c7bcf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:18.888479 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d24185d-e77a-4567-929f-bc16f62c7bcf None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:18.899532 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544c737a-8085-4664-84df-65cd1ca4a5f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:18.900446 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544c737a-8085-4664-84df-65cd1ca4a5f9 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:18.900916 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544c737a-8085-4664-84df-65cd1ca4a5f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:18.901413 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544c737a-8085-4664-84df-65cd1ca4a5f9 None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:18.902340 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544c737a-8085-4664-84df-65cd1ca4a5f9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:18.902848 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544c737a-8085-4664-84df-65cd1ca4a5f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:18.904392 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544c737a-8085-4664-84df-65cd1ca4a5f9 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:18.912717 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 286/552] 10.222.0.102 () {56 vars in 1140 bytes} [Thu Sep 1 17:18:18 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 372 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:18:18.914223 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534c9073-82a8-403e-b85d-8f0538b8f7ed None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:18.914509 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534c9073-82a8-403e-b85d-8f0538b8f7ed None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:18.914782 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534c9073-82a8-403e-b85d-8f0538b8f7ed None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:18.915730 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 267/553] 10.222.0.102 () {56 vars in 1108 bytes} [Thu Sep 1 17:18:18 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:18:18.920531 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 287/554] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:18:18 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:18:18.920672 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544c737a-8085-4664-84df-65cd1ca4a5f9 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:18.920761 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534c9073-82a8-403e-b85d-8f0538b8f7ed None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:19.013696 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:19.029774 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544c737a-8085-4664-84df-65cd1ca4a5f9 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:19.040534 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534c9073-82a8-403e-b85d-8f0538b8f7ed None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:19.065856 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713b5262-a755-4f1f-9889-e95709ffbde1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:19.067031 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713b5262-a755-4f1f-9889-e95709ffbde1 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:19.067322 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713b5262-a755-4f1f-9889-e95709ffbde1 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:19.067668 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713b5262-a755-4f1f-9889-e95709ffbde1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:19.080335 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e331a23-da62-4e75-bb1e-4a468e570709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:19.081275 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e331a23-da62-4e75-bb1e-4a468e570709 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:19.081796 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e331a23-da62-4e75-bb1e-4a468e570709 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:19.082289 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e331a23-da62-4e75-bb1e-4a468e570709 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:19.094650 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e331a23-da62-4e75-bb1e-4a468e570709 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:19.096528 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e331a23-da62-4e75-bb1e-4a468e570709 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:19.114318 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-4e331a23-da62-4e75-bb1e-4a468e570709 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:19.114318 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:19.114318 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:19.114318 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:19.114318 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:19.114318 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:19.114318 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:19.114318 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:19.114318 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:19.114318 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:19.114318 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:19.114318 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:19.114318 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:18:19.114318 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:18:19.114318 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:18:19.114318 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:18:19.115263 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:19.115263 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:19.115263 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:19.115263 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:19.115263 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:19.115263 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:19.115263 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:19.115263 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:19.115263 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:19.115263 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:19.115263 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:19.115263 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:19.115263 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:19.115263 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:19.115263 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:19.115263 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:19.115263 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:18:19.116183 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:18:19.116183 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:19.116183 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:18:19.118344 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 288/555] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:18:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:19.119721 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5fde7f-eca3-40f0-a180-57956a49b393 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:19.120213 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5fde7f-eca3-40f0-a180-57956a49b393 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:19.120687 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5fde7f-eca3-40f0-a180-57956a49b393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:19.141149 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-3e5fde7f-eca3-40f0-a180-57956a49b393 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:18:19.142884 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-3e5fde7f-eca3-40f0-a180-57956a49b393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:18:19.146117 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713b5262-a755-4f1f-9889-e95709ffbde1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:19.148476 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713b5262-a755-4f1f-9889-e95709ffbde1 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:19.171437 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:19.229067 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 289/556] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:18:19 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:18:19.229067 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5fde7f-eca3-40f0-a180-57956a49b393 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:19.250760 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:19.274728 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 268/557] 10.222.0.102 () {58 vars in 1287 bytes} [Thu Sep 1 17:18:18 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 01 17:18:19.276366 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da0b899-054b-49b2-a33a-af2f42be12e0 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:19.276900 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da0b899-054b-49b2-a33a-af2f42be12e0 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:19.277177 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da0b899-054b-49b2-a33a-af2f42be12e0 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:19.279106 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 269/558] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:18:19 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:18:19.279887 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da0b899-054b-49b2-a33a-af2f42be12e0 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:19.337572 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5fde7f-eca3-40f0-a180-57956a49b393 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:19.374063 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04901c9-17a5-443b-86d7-8df576c37202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:19.374790 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04901c9-17a5-443b-86d7-8df576c37202 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:19.375067 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04901c9-17a5-443b-86d7-8df576c37202 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:19.375336 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04901c9-17a5-443b-86d7-8df576c37202 None admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:19.376090 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04901c9-17a5-443b-86d7-8df576c37202 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:19.376390 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04901c9-17a5-443b-86d7-8df576c37202 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:19.377807 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04901c9-17a5-443b-86d7-8df576c37202 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:19.393457 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 290/559] 10.222.0.102 () {56 vars in 1114 bytes} [Thu Sep 1 17:18:19 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:19.396839 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb79bd6d-9914-407b-abab-9a5d036fda50 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:19.397504 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb79bd6d-9914-407b-abab-9a5d036fda50 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:19.397783 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb79bd6d-9914-407b-abab-9a5d036fda50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:19.399859 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da0b899-054b-49b2-a33a-af2f42be12e0 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:19.416153 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-fb79bd6d-9914-407b-abab-9a5d036fda50 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:18:19.417686 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-fb79bd6d-9914-407b-abab-9a5d036fda50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:18:19.431944 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:19.436219 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb21f84-4d74-4a50-81c7-7746238e1e77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:19.437061 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb21f84-4d74-4a50-81c7-7746238e1e77 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:19.437362 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb21f84-4d74-4a50-81c7-7746238e1e77 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:19.437931 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb21f84-4d74-4a50-81c7-7746238e1e77 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:19.449067 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb21f84-4d74-4a50-81c7-7746238e1e77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:19.450557 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb21f84-4d74-4a50-81c7-7746238e1e77 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:19.465923 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-abb21f84-4d74-4a50-81c7-7746238e1e77 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:19.465923 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:19.465923 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:19.465923 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:19.465923 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:19.465923 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:19.465923 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:19.465923 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:19.465923 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:19.465923 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:19.465923 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:19.465923 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:19.465923 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:18:19.465923 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:18:19.465923 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:18:19.465923 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:18:19.466861 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:19.466861 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:19.466861 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:19.466861 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:19.466861 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:19.466861 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:19.466861 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:19.466861 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:19.466861 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:19.466861 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:19.466861 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:19.466861 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:19.466861 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:19.466861 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:19.466861 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:19.466861 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:19.466861 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:18:19.467717 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:18:19.467717 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:19.467717 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:18:19.467717 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 270/560] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:18:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:19.468636 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014c8537-3fb8-4568-81eb-aa26714b0cd7 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:19.468636 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014c8537-3fb8-4568-81eb-aa26714b0cd7 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:19.468827 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014c8537-3fb8-4568-81eb-aa26714b0cd7 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:19.470208 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 271/561] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:18:19 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:18:19.474433 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014c8537-3fb8-4568-81eb-aa26714b0cd7 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:19.502062 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 291/562] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:18:19 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:18:19.502062 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb79bd6d-9914-407b-abab-9a5d036fda50 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:19.525803 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:19.593920 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014c8537-3fb8-4568-81eb-aa26714b0cd7 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:19.604058 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb79bd6d-9914-407b-abab-9a5d036fda50 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:19.631709 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ec547b-df2c-406b-862c-37325ec5d24d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:19.632346 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ec547b-df2c-406b-862c-37325ec5d24d None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:19.632611 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ec547b-df2c-406b-862c-37325ec5d24d None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:19.632882 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ec547b-df2c-406b-862c-37325ec5d24d None admin] PATH_INFO: `/v3/users/nova` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:19.639591 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55155cd8-07b8-41f9-b6b4-d18b36f81963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:19.640185 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55155cd8-07b8-41f9-b6b4-d18b36f81963 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:19.640461 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55155cd8-07b8-41f9-b6b4-d18b36f81963 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:19.640730 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55155cd8-07b8-41f9-b6b4-d18b36f81963 None admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:19.640825 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ec547b-df2c-406b-862c-37325ec5d24d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:19.641393 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55155cd8-07b8-41f9-b6b4-d18b36f81963 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:19.641696 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55155cd8-07b8-41f9-b6b4-d18b36f81963 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:19.642202 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ec547b-df2c-406b-862c-37325ec5d24d None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:19.643029 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55155cd8-07b8-41f9-b6b4-d18b36f81963 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:19.653824 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-29ec547b-df2c-406b-862c-37325ec5d24d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 17:18:19.653824 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:19.653824 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:19.653824 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:19.653824 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:19.653824 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:19.653824 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:19.653824 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:19.653824 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:19.653824 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:19.653824 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:19.653824 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:19.653824 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:18:19.653824 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:18:19.653824 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:18:19.653824 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:18:19.659723 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:19.659723 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:19.659723 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:18:19.659723 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:18:19.659723 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:18:19.659723 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:18:19.659723 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:19.659723 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:19.659723 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:19.659723 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:19.659723 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:19.659723 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:19.659723 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:19.659723 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:19.659723 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:19.659723 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:19.659723 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:19.659723 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:19.660837 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:19.660837 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:19.660837 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:18:19.660837 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:18:19.660837 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:18:19.660837 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:18:19.660837 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:18:19.660837 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:18:19.660837 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 17:18:19.660837 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:18:19.661601 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 292/563] 10.222.0.102 () {56 vars in 1114 bytes} [Thu Sep 1 17:18:19 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:19.661685 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 272/564] 10.222.0.102 () {56 vars in 1098 bytes} [Thu Sep 1 17:18:19 2022] GET /identity/v3/users/nova => generated 82 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:19.661685 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ec547b-df2c-406b-862c-37325ec5d24d None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:19.661828 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f527e58e-72fa-499d-abec-9b543d340d5f None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:19.661828 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f527e58e-72fa-499d-abec-9b543d340d5f None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:19.661828 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f527e58e-72fa-499d-abec-9b543d340d5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:19.677653 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-f527e58e-72fa-499d-abec-9b543d340d5f None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:18:19.678834 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-f527e58e-72fa-499d-abec-9b543d340d5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:18:19.680968 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:19.757478 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 293/565] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:18:19 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:18:19.757478 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f527e58e-72fa-499d-abec-9b543d340d5f None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:19.778564 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:19.783398 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ec547b-df2c-406b-862c-37325ec5d24d None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:19.821933 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe2beb6-e57a-4350-94c3-fc7df101e11c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:19.822508 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe2beb6-e57a-4350-94c3-fc7df101e11c None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:19.822789 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe2beb6-e57a-4350-94c3-fc7df101e11c None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:19.823065 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe2beb6-e57a-4350-94c3-fc7df101e11c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:19.833328 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe2beb6-e57a-4350-94c3-fc7df101e11c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:18:19.838647 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-8fe2beb6-e57a-4350-94c3-fc7df101e11c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:18:19.838647 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:19.838647 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:19.838647 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:19.838647 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:19.838647 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:19.838647 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:19.838647 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:19.838647 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:19.838647 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:19.838647 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:19.838647 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:19.838647 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:18:19.838647 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:18:19.838647 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:18:19.838647 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:18:19.840641 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:18:19.840641 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:18:19.840641 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:18:19.840641 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:18:19.840641 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:19.840641 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:19.840641 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:19.840641 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:19.840641 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:19.840641 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:19.840641 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:19.840641 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:19.840641 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:19.840641 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:19.840641 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:19.840641 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:19.842224 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:19.842224 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:19.842224 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:19.842224 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:19.842224 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:18:19.842224 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:18:19.842224 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:18:19.842224 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:18:19.842224 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:18:19.842224 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:18:19.842224 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:18:19.842224 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:18:19.842224 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 273/566] 10.222.0.102 () {56 vars in 1107 bytes} [Thu Sep 1 17:18:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:19.842224 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe2beb6-e57a-4350-94c3-fc7df101e11c None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:19.875531 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f527e58e-72fa-499d-abec-9b543d340d5f None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:19.913932 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7620ba6-8536-4cd2-be90-3a9f74ccba37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:19.914576 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7620ba6-8536-4cd2-be90-3a9f74ccba37 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:19.914839 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7620ba6-8536-4cd2-be90-3a9f74ccba37 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:19.915103 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7620ba6-8536-4cd2-be90-3a9f74ccba37 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:19.922481 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7620ba6-8536-4cd2-be90-3a9f74ccba37 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:19.924052 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7620ba6-8536-4cd2-be90-3a9f74ccba37 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:19.926516 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:19.938164 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-a7620ba6-8536-4cd2-be90-3a9f74ccba37 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:18:19.938164 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:19.938164 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:19.938164 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:19.938164 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:19.938164 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:19.938164 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:19.938164 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:19.938164 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:19.938164 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:19.938164 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:19.938164 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:19.938164 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:18:19.938164 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:18:19.938164 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:18:19.939116 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:18:19.939116 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:19.939116 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:19.939116 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:19.939116 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:19.939116 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:19.939116 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:19.939116 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:19.939116 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:19.939116 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:19.939116 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:19.939116 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:19.939116 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:19.939116 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:19.939116 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:19.939116 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:19.939116 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:19.940041 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:18:19.940041 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:18:19.940041 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:18:19.940041 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:18:19.940041 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:18:19.940041 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:18:19.940041 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:18:19.940041 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:18:19.940041 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 294/567] 10.222.0.102 () {56 vars in 1116 bytes} [Thu Sep 1 17:18:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:19.940041 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7620ba6-8536-4cd2-be90-3a9f74ccba37 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:19.960219 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe2beb6-e57a-4350-94c3-fc7df101e11c None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:19.997237 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6947d35-5e7c-448f-b839-9f4bb5011886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:19.997873 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6947d35-5e7c-448f-b839-9f4bb5011886 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:19.998155 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6947d35-5e7c-448f-b839-9f4bb5011886 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:19.998433 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6947d35-5e7c-448f-b839-9f4bb5011886 None admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:19.999163 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6947d35-5e7c-448f-b839-9f4bb5011886 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:19.999475 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6947d35-5e7c-448f-b839-9f4bb5011886 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:20.000916 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6947d35-5e7c-448f-b839-9f4bb5011886 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:20.019153 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 274/568] 10.222.0.102 () {56 vars in 1138 bytes} [Thu Sep 1 17:18:19 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 370 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:20.019794 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6947d35-5e7c-448f-b839-9f4bb5011886 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:20.039041 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:20.055658 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7620ba6-8536-4cd2-be90-3a9f74ccba37 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:20.089918 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238bae25-cb9c-4544-968f-9eee7f77e59f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:20.090543 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238bae25-cb9c-4544-968f-9eee7f77e59f None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:20.090822 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238bae25-cb9c-4544-968f-9eee7f77e59f None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:20.091090 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238bae25-cb9c-4544-968f-9eee7f77e59f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:20.103185 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238bae25-cb9c-4544-968f-9eee7f77e59f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:18:20.108388 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-238bae25-cb9c-4544-968f-9eee7f77e59f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:18:20.108388 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:20.108388 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:20.108388 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:20.108388 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:20.108388 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:20.108388 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:20.108388 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:20.108388 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:20.108388 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:20.108388 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:20.108388 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:20.108388 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:18:20.108388 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:18:20.108388 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:18:20.108388 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:18:20.109502 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:18:20.109502 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:18:20.109502 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:18:20.109502 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:18:20.109502 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:20.109502 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:20.109502 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:20.109502 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:20.109502 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:20.109502 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:20.109502 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:20.109502 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:20.109502 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:20.109502 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:20.109502 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:20.109502 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:20.110519 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:20.110519 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:20.110519 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:20.110519 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:20.110519 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:18:20.110519 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:18:20.110519 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:18:20.110519 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:18:20.110519 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:18:20.110519 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:18:20.110519 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:18:20.110519 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:18:20.110519 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 295/569] 10.222.0.102 () {56 vars in 1107 bytes} [Thu Sep 1 17:18:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:20.110519 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238bae25-cb9c-4544-968f-9eee7f77e59f None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:20.125319 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6947d35-5e7c-448f-b839-9f4bb5011886 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:20.158720 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fec48e-fd23-4ef0-955d-a641ba8d578b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:20.159370 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fec48e-fd23-4ef0-955d-a641ba8d578b None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:20.159638 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fec48e-fd23-4ef0-955d-a641ba8d578b None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:20.159905 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fec48e-fd23-4ef0-955d-a641ba8d578b None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:20.161182 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fec48e-fd23-4ef0-955d-a641ba8d578b None admin] RBAC: Adding query filter params (name=service) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:20.161544 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fec48e-fd23-4ef0-955d-a641ba8d578b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:20.162866 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fec48e-fd23-4ef0-955d-a641ba8d578b None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:20.169293 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:20.170725 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 275/570] 10.222.0.102 () {56 vars in 1108 bytes} [Thu Sep 1 17:18:20 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:20.176401 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fec48e-fd23-4ef0-955d-a641ba8d578b None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:20.211698 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238bae25-cb9c-4544-968f-9eee7f77e59f None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:20.246610 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99fc8ce-22e1-46df-9e6f-a48dae3220bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:20.247211 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99fc8ce-22e1-46df-9e6f-a48dae3220bf None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:20.247496 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99fc8ce-22e1-46df-9e6f-a48dae3220bf None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:20.247770 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99fc8ce-22e1-46df-9e6f-a48dae3220bf None admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:20.248453 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99fc8ce-22e1-46df-9e6f-a48dae3220bf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:20.248760 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99fc8ce-22e1-46df-9e6f-a48dae3220bf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:20.250212 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99fc8ce-22e1-46df-9e6f-a48dae3220bf None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:20.253417 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:20.262109 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 296/571] 10.222.0.102 () {56 vars in 1153 bytes} [Thu Sep 1 17:18:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:20.262906 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99fc8ce-22e1-46df-9e6f-a48dae3220bf None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:20.278097 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fec48e-fd23-4ef0-955d-a641ba8d578b None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:20.311499 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9050455f-892b-40d5-80d5-10fc8549e68b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:20.312127 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9050455f-892b-40d5-80d5-10fc8549e68b None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:20.312409 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9050455f-892b-40d5-80d5-10fc8549e68b None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:20.312686 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9050455f-892b-40d5-80d5-10fc8549e68b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:20.323029 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9050455f-892b-40d5-80d5-10fc8549e68b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:20.324474 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9050455f-892b-40d5-80d5-10fc8549e68b None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:20.339918 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-9050455f-892b-40d5-80d5-10fc8549e68b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:20.339918 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:20.339918 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:20.339918 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:20.339918 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:20.339918 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:20.339918 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:20.339918 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:20.339918 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:20.339918 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:20.339918 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:20.339918 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:20.339918 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:18:20.339918 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:18:20.339918 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:18:20.339918 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:18:20.340855 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:20.340855 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:20.340855 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:20.340855 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:20.340855 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:20.340855 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:20.340855 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:20.340855 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:20.340855 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:20.340855 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:20.340855 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:20.340855 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:20.340855 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:20.340855 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:20.340855 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:20.340855 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:20.340855 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:18:20.341768 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:18:20.341768 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:20.341768 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:18:20.341768 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 276/572] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:18:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:20.341994 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9050455f-892b-40d5-80d5-10fc8549e68b None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:20.364589 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99fc8ce-22e1-46df-9e6f-a48dae3220bf None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:20.382237 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:20.400173 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efed4eb8-d4a7-4bab-92fc-19cb98ddf81d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:20.401121 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efed4eb8-d4a7-4bab-92fc-19cb98ddf81d None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:20.401508 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efed4eb8-d4a7-4bab-92fc-19cb98ddf81d None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:20.401786 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efed4eb8-d4a7-4bab-92fc-19cb98ddf81d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:20.412642 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efed4eb8-d4a7-4bab-92fc-19cb98ddf81d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:18:20.417400 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-efed4eb8-d4a7-4bab-92fc-19cb98ddf81d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:18:20.417400 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:20.417400 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:20.417400 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:20.417400 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:20.417400 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:20.417400 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:20.417400 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:20.417400 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:20.417400 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:20.417400 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:20.417400 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:20.417400 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:18:20.417400 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:18:20.417400 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:18:20.417400 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:18:20.418506 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:18:20.418506 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:18:20.418506 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:18:20.418506 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:18:20.418506 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:20.418506 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:20.418506 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:20.418506 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:20.418506 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:20.418506 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:20.418506 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:20.418506 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:20.418506 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:20.418506 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:20.418506 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:20.418506 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:20.419580 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:20.419580 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:20.419580 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:20.419580 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:20.419580 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:18:20.419580 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:18:20.419580 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:18:20.419580 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:18:20.419580 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:18:20.419580 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:18:20.419580 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:18:20.419580 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:18:20.419580 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 297/573] 10.222.0.102 () {56 vars in 1107 bytes} [Thu Sep 1 17:18:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:20.419580 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efed4eb8-d4a7-4bab-92fc-19cb98ddf81d None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:20.445426 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9050455f-892b-40d5-80d5-10fc8549e68b None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:20.473328 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:20.479575 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea9f174-b163-4a31-95d4-27b8f56f422e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:20.480174 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea9f174-b163-4a31-95d4-27b8f56f422e None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:20.480452 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea9f174-b163-4a31-95d4-27b8f56f422e None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:20.480729 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea9f174-b163-4a31-95d4-27b8f56f422e None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:20.481414 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea9f174-b163-4a31-95d4-27b8f56f422e None admin] RBAC: Adding query filter params (name=service) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:20.481723 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea9f174-b163-4a31-95d4-27b8f56f422e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:20.483075 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea9f174-b163-4a31-95d4-27b8f56f422e None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:20.491241 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 277/574] 10.222.0.102 () {56 vars in 1108 bytes} [Thu Sep 1 17:18:20 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:20.491848 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea9f174-b163-4a31-95d4-27b8f56f422e None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:20.519016 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efed4eb8-d4a7-4bab-92fc-19cb98ddf81d None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:20.554416 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb98adc-277b-45a3-b44c-b922c294e5df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:20.555041 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb98adc-277b-45a3-b44c-b922c294e5df None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:20.555323 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb98adc-277b-45a3-b44c-b922c294e5df None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:20.555595 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb98adc-277b-45a3-b44c-b922c294e5df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:20.565484 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb98adc-277b-45a3-b44c-b922c294e5df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:20.567097 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb98adc-277b-45a3-b44c-b922c294e5df None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:20.581858 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-ddb98adc-277b-45a3-b44c-b922c294e5df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:20.581858 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:20.581858 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:20.581858 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:20.581858 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:20.581858 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:20.581858 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:20.581858 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:20.581858 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:20.581858 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:20.581858 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:20.581858 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:20.581858 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:18:20.581858 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:18:20.581858 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:18:20.581858 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:18:20.583375 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:20.583375 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:20.583375 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:20.583375 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:20.583375 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:20.583375 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:20.583375 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:20.583375 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:20.583375 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:20.583375 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:20.583375 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:20.583375 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:20.583375 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:20.583375 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:20.583375 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:20.583375 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:20.583375 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:18:20.584278 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:18:20.584278 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:20.584278 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:18:20.584278 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 298/575] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:18:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:20.584278 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb98adc-277b-45a3-b44c-b922c294e5df None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:20.596210 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea9f174-b163-4a31-95d4-27b8f56f422e None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:20.614731 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:20.631523 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d910cf-a6be-4140-8def-67f2f1acb333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:20.632132 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d910cf-a6be-4140-8def-67f2f1acb333 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:20.632412 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d910cf-a6be-4140-8def-67f2f1acb333 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:20.632693 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d910cf-a6be-4140-8def-67f2f1acb333 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:20.633433 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d910cf-a6be-4140-8def-67f2f1acb333 None admin] RBAC: Adding query filter params (role.id=f21e23654e4740158c7d4f46e726e515, scope.project.id=e78fff7f1f6e4f68b8d233ad2e5bbd4f, user.id=d55edf6cd1804006aa8269b4c6f80aec) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:20.633742 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d910cf-a6be-4140-8def-67f2f1acb333 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:20.635178 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d910cf-a6be-4140-8def-67f2f1acb333 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:20.640526 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 278/576] 10.222.0.102 () {56 vars in 1379 bytes} [Thu Sep 1 17:18:20 2022] GET /identity/v3/role_assignments?user.id=d55edf6cd1804006aa8269b4c6f80aec&scope.project.id=e78fff7f1f6e4f68b8d233ad2e5bbd4f&role.id=f21e23654e4740158c7d4f46e726e515 => generated 260 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:20.641388 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d910cf-a6be-4140-8def-67f2f1acb333 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:20.695536 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb98adc-277b-45a3-b44c-b922c294e5df None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:20.714337 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:20.735410 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01c6797-e17d-4285-8660-21cdbeeff10c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:20.736027 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01c6797-e17d-4285-8660-21cdbeeff10c None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:20.736308 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01c6797-e17d-4285-8660-21cdbeeff10c None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:20.736582 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01c6797-e17d-4285-8660-21cdbeeff10c None admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:20.737235 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01c6797-e17d-4285-8660-21cdbeeff10c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:20.737576 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01c6797-e17d-4285-8660-21cdbeeff10c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:20.739050 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01c6797-e17d-4285-8660-21cdbeeff10c None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:20.756352 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 299/577] 10.222.0.102 () {56 vars in 1114 bytes} [Thu Sep 1 17:18:20 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:20.756352 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01c6797-e17d-4285-8660-21cdbeeff10c None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:20.771777 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d910cf-a6be-4140-8def-67f2f1acb333 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:20.814870 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea6dd84-6ba6-4bf6-967f-c42a0563f254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:20.815566 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea6dd84-6ba6-4bf6-967f-c42a0563f254 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:20.815918 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea6dd84-6ba6-4bf6-967f-c42a0563f254 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:20.816260 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea6dd84-6ba6-4bf6-967f-c42a0563f254 None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:20.817580 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea6dd84-6ba6-4bf6-967f-c42a0563f254 None admin] RBAC: Adding query filter params (name=service) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:20.817969 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea6dd84-6ba6-4bf6-967f-c42a0563f254 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:20.819450 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea6dd84-6ba6-4bf6-967f-c42a0563f254 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:20.828306 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 279/578] 10.222.0.102 () {56 vars in 1108 bytes} [Thu Sep 1 17:18:20 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:20.829267 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea6dd84-6ba6-4bf6-967f-c42a0563f254 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:20.866143 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:20.879387 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01c6797-e17d-4285-8660-21cdbeeff10c None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:20.917800 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f90e6b1-f35d-4e59-b536-c915e88cd300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:20.918436 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f90e6b1-f35d-4e59-b536-c915e88cd300 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:20.918751 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f90e6b1-f35d-4e59-b536-c915e88cd300 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:20.919140 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f90e6b1-f35d-4e59-b536-c915e88cd300 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:20.931808 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f90e6b1-f35d-4e59-b536-c915e88cd300 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:20.933269 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f90e6b1-f35d-4e59-b536-c915e88cd300 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:20.948467 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-1f90e6b1-f35d-4e59-b536-c915e88cd300 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:20.948467 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:20.948467 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:20.948467 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:20.948467 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:20.948467 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:20.948467 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:20.948467 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:20.948467 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:20.948467 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:20.948467 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:20.948467 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:20.948467 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:18:20.948467 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:18:20.948467 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:18:20.948467 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:18:20.949479 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:20.949479 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:20.949479 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:20.949479 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:20.949479 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:20.949479 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:20.949479 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:20.949479 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:20.949479 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:20.949479 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:20.949479 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:20.949479 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:20.949479 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:20.949479 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:20.949479 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:20.949479 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:20.949479 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:18:20.950349 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:18:20.950349 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:20.950349 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:18:20.950517 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea6dd84-6ba6-4bf6-967f-c42a0563f254 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:20.954406 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 300/579] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:18:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:20.955246 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f90e6b1-f35d-4e59-b536-c915e88cd300 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:20.970051 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:20.991172 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b4351a-6054-4b2a-903b-05d2882262ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:20.991789 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b4351a-6054-4b2a-903b-05d2882262ec None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:20.992072 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b4351a-6054-4b2a-903b-05d2882262ec None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:20.992355 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b4351a-6054-4b2a-903b-05d2882262ec None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:20.993001 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b4351a-6054-4b2a-903b-05d2882262ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:20.993315 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b4351a-6054-4b2a-903b-05d2882262ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:20.994737 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b4351a-6054-4b2a-903b-05d2882262ec None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:21.010791 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 280/580] 10.222.0.102 () {56 vars in 1114 bytes} [Thu Sep 1 17:18:20 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:21.010791 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b4351a-6054-4b2a-903b-05d2882262ec None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:21.069211 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f90e6b1-f35d-4e59-b536-c915e88cd300 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:21.110468 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719dc313-2f5c-4075-bc78-8e5371644bf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:21.111095 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719dc313-2f5c-4075-bc78-8e5371644bf4 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:21.111376 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719dc313-2f5c-4075-bc78-8e5371644bf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:21.111646 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719dc313-2f5c-4075-bc78-8e5371644bf4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:21.114810 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b4351a-6054-4b2a-903b-05d2882262ec None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:21.118588 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719dc313-2f5c-4075-bc78-8e5371644bf4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:21.120107 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719dc313-2f5c-4075-bc78-8e5371644bf4 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:21.131771 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-719dc313-2f5c-4075-bc78-8e5371644bf4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:18:21.131771 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:21.131771 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:21.131771 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:21.131771 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:21.131771 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:21.131771 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:21.131771 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:21.131771 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:21.131771 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:21.131771 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:21.131771 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:21.131771 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:18:21.131771 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:18:21.131771 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:18:21.132699 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:18:21.132699 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:21.132699 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:21.132699 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:21.132699 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:21.132699 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:21.132699 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:21.132699 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:21.132699 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:21.132699 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:21.132699 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:21.132699 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:21.132699 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:21.132699 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:21.132699 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:21.132699 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:21.132699 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:21.133676 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:18:21.133676 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:18:21.133676 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:18:21.133676 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:18:21.133676 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:18:21.133676 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:18:21.133676 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:18:21.133676 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:18:21.133676 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 301/581] 10.222.0.102 () {56 vars in 1116 bytes} [Thu Sep 1 17:18:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:21.134202 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719dc313-2f5c-4075-bc78-8e5371644bf4 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:21.137243 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:21.158524 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036c94cd-fd69-4f13-8404-2da722ec5815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:21.159167 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036c94cd-fd69-4f13-8404-2da722ec5815 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:21.159972 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036c94cd-fd69-4f13-8404-2da722ec5815 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:21.160903 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036c94cd-fd69-4f13-8404-2da722ec5815 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:21.171316 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036c94cd-fd69-4f13-8404-2da722ec5815 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:21.172768 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036c94cd-fd69-4f13-8404-2da722ec5815 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:21.187626 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-036c94cd-fd69-4f13-8404-2da722ec5815 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:21.187626 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:21.187626 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:21.187626 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:21.187626 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:21.187626 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:21.187626 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:21.187626 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:21.187626 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:21.187626 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:21.187626 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:21.187626 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:21.187626 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:18:21.187626 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:18:21.187626 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:18:21.187626 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:18:21.188758 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:21.188758 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:21.188758 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:21.188758 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:21.188758 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:21.188758 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:21.188758 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:21.188758 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:21.188758 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:21.188758 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:21.188758 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:21.188758 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:21.188758 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:21.188758 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:21.188758 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:21.188758 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:21.188758 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:18:21.189794 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:18:21.189794 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:21.189794 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:18:21.189794 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 281/582] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:18:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:21.189794 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036c94cd-fd69-4f13-8404-2da722ec5815 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:21.222332 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:21.255310 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719dc313-2f5c-4075-bc78-8e5371644bf4 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:21.295080 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad97e01f-db69-4844-989d-454ebdec6f99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:21.295287 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036c94cd-fd69-4f13-8404-2da722ec5815 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:21.295689 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad97e01f-db69-4844-989d-454ebdec6f99 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:21.295972 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad97e01f-db69-4844-989d-454ebdec6f99 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:21.296244 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad97e01f-db69-4844-989d-454ebdec6f99 None admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:21.297071 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad97e01f-db69-4844-989d-454ebdec6f99 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:21.298170 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad97e01f-db69-4844-989d-454ebdec6f99 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:21.299573 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad97e01f-db69-4844-989d-454ebdec6f99 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:21.311533 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 302/583] 10.222.0.102 () {56 vars in 1114 bytes} [Thu Sep 1 17:18:21 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:21.312339 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad97e01f-db69-4844-989d-454ebdec6f99 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:21.339894 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cd0dbf-e3ac-4c60-b4a6-2c2d8078baa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:21.340451 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cd0dbf-e3ac-4c60-b4a6-2c2d8078baa0 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:21.340722 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cd0dbf-e3ac-4c60-b4a6-2c2d8078baa0 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:21.340998 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cd0dbf-e3ac-4c60-b4a6-2c2d8078baa0 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:21.349392 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cd0dbf-e3ac-4c60-b4a6-2c2d8078baa0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:21.350850 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cd0dbf-e3ac-4c60-b4a6-2c2d8078baa0 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:21.364380 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-07cd0dbf-e3ac-4c60-b4a6-2c2d8078baa0 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 01 17:18:21.364380 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:21.364380 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:21.364380 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:21.364380 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:21.364380 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:21.364380 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:21.364380 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:21.364380 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:21.364380 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:21.364380 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:21.364380 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:21.364380 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:18:21.364380 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:18:21.364380 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:18:21.364380 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:18:21.365352 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:21.365352 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:21.365352 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:18:21.365352 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:18:21.365352 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:18:21.365352 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:18:21.365352 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:21.365352 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:21.365352 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:21.365352 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:21.365352 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:21.365352 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:21.365352 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:21.365352 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:21.365352 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:21.365352 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:21.365352 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:21.365352 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:21.366306 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:21.366306 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:21.366306 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:18:21.366306 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:18:21.366306 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:18:21.366306 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:18:21.366306 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:18:21.366306 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:18:21.366306 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 01 17:18:21.366306 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:18:21.366306 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 282/584] 10.222.0.102 () {56 vars in 1104 bytes} [Thu Sep 1 17:18:21 2022] GET /identity/v3/users/glance => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:21.366306 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cd0dbf-e3ac-4c60-b4a6-2c2d8078baa0 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:21.387800 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:21.430947 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad97e01f-db69-4844-989d-454ebdec6f99 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:21.466082 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cb40a6-7ad3-4542-84c0-f8bea1382cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:21.466689 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cb40a6-7ad3-4542-84c0-f8bea1382cd4 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:21.466969 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cb40a6-7ad3-4542-84c0-f8bea1382cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:21.467070 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:21.467240 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cb40a6-7ad3-4542-84c0-f8bea1382cd4 None admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:21.467932 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cb40a6-7ad3-4542-84c0-f8bea1382cd4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:21.468234 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cb40a6-7ad3-4542-84c0-f8bea1382cd4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:21.469675 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cb40a6-7ad3-4542-84c0-f8bea1382cd4 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:21.480575 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cd0dbf-e3ac-4c60-b4a6-2c2d8078baa0 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:21.483015 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 303/585] 10.222.0.102 () {56 vars in 1153 bytes} [Thu Sep 1 17:18:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:21.483925 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cb40a6-7ad3-4542-84c0-f8bea1382cd4 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:21.519694 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4836c8e3-fe6b-4aba-814e-ed09d49476ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:21.520301 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4836c8e3-fe6b-4aba-814e-ed09d49476ca None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:21.520581 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4836c8e3-fe6b-4aba-814e-ed09d49476ca None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:21.520851 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4836c8e3-fe6b-4aba-814e-ed09d49476ca None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:21.521531 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4836c8e3-fe6b-4aba-814e-ed09d49476ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:21.521914 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4836c8e3-fe6b-4aba-814e-ed09d49476ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:21.523424 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4836c8e3-fe6b-4aba-814e-ed09d49476ca None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:21.536813 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 283/586] 10.222.0.102 () {56 vars in 1114 bytes} [Thu Sep 1 17:18:21 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:21.537653 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4836c8e3-fe6b-4aba-814e-ed09d49476ca None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:21.587418 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cb40a6-7ad3-4542-84c0-f8bea1382cd4 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:21.612679 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:21.625676 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf36aaf-0412-4087-be45-c56a5b32c97b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:21.626312 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf36aaf-0412-4087-be45-c56a5b32c97b None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:21.626592 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf36aaf-0412-4087-be45-c56a5b32c97b None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:21.626864 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf36aaf-0412-4087-be45-c56a5b32c97b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:21.634833 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf36aaf-0412-4087-be45-c56a5b32c97b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:21.636301 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf36aaf-0412-4087-be45-c56a5b32c97b None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:21.646105 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4836c8e3-fe6b-4aba-814e-ed09d49476ca None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:21.647595 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-8cf36aaf-0412-4087-be45-c56a5b32c97b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 17:18:21.647595 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:21.647595 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:21.647595 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:21.647595 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:21.647595 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:21.647595 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:21.647595 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:21.647595 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:21.647595 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:21.647595 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:21.647595 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:21.647595 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:18:21.647595 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:18:21.647595 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:18:21.647595 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:18:21.648614 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:21.648614 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:21.648614 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:18:21.648614 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:18:21.648614 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:18:21.648614 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:18:21.648614 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:21.648614 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:21.648614 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:21.648614 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:21.648614 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:21.648614 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:21.648614 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:21.648614 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:21.648614 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:21.648614 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:21.648614 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:21.648614 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:21.649596 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:21.649596 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:21.649596 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:18:21.649596 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:18:21.649596 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:18:21.649596 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:18:21.649596 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:18:21.649596 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:18:21.649596 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 17:18:21.649596 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:18:21.653562 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 304/587] 10.222.0.102 () {56 vars in 1104 bytes} [Thu Sep 1 17:18:21 2022] GET /identity/v3/users/cinder => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:21.654238 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf36aaf-0412-4087-be45-c56a5b32c97b None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:21.688712 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5496c345-121f-40cf-bca0-91da7004cf1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:21.689341 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5496c345-121f-40cf-bca0-91da7004cf1b None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:21.689785 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5496c345-121f-40cf-bca0-91da7004cf1b None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:21.690057 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5496c345-121f-40cf-bca0-91da7004cf1b None admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:21.691614 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5496c345-121f-40cf-bca0-91da7004cf1b None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:21.691923 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5496c345-121f-40cf-bca0-91da7004cf1b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:21.693462 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5496c345-121f-40cf-bca0-91da7004cf1b None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:21.710934 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 284/588] 10.222.0.102 () {56 vars in 1142 bytes} [Thu Sep 1 17:18:21 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 374 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:21.711781 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5496c345-121f-40cf-bca0-91da7004cf1b None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:21.726472 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:21.765153 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf36aaf-0412-4087-be45-c56a5b32c97b None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:21.806262 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3971da3a-0f0f-4f1a-9fab-e3c97f403003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:21.807042 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3971da3a-0f0f-4f1a-9fab-e3c97f403003 None admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:21.807325 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3971da3a-0f0f-4f1a-9fab-e3c97f403003 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:21.807599 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3971da3a-0f0f-4f1a-9fab-e3c97f403003 None admin] PATH_INFO: `/v3/projects/e78fff7f1f6e4f68b8d233ad2e5bbd4f/users/5f8293316baf4774b62fc11d55d50865/roles/f21e23654e4740158c7d4f46e726e515` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:21.823538 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3971da3a-0f0f-4f1a-9fab-e3c97f403003 None admin] RBAC: Authorizing `identity:create_grant(project_id=e78fff7f1f6e4f68b8d233ad2e5bbd4f, user_id=5f8293316baf4774b62fc11d55d50865, role_id=f21e23654e4740158c7d4f46e726e515)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:21.824957 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3971da3a-0f0f-4f1a-9fab-e3c97f403003 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:21.828787 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5496c345-121f-40cf-bca0-91da7004cf1b None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:21.849333 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 305/589] 10.222.0.102 () {58 vars in 1444 bytes} [Thu Sep 1 17:18:21 2022] PUT /identity/v3/projects/e78fff7f1f6e4f68b8d233ad2e5bbd4f/users/5f8293316baf4774b62fc11d55d50865/roles/f21e23654e4740158c7d4f46e726e515 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:18:21.849333 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3971da3a-0f0f-4f1a-9fab-e3c97f403003 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:21.891248 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5915418-3242-4dae-87d1-967c94dc813b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:21.892409 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:21.893632 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5915418-3242-4dae-87d1-967c94dc813b None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:21.894055 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5915418-3242-4dae-87d1-967c94dc813b None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:21.895118 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5915418-3242-4dae-87d1-967c94dc813b None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:21.907091 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5915418-3242-4dae-87d1-967c94dc813b None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:21.910693 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5915418-3242-4dae-87d1-967c94dc813b None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:21.926521 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-c5915418-3242-4dae-87d1-967c94dc813b None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 01 17:18:21.926521 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:21.926521 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:21.926521 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:21.926521 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:21.926521 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:21.926521 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:21.926521 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:21.926521 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:21.926521 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:21.926521 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:21.926521 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:21.926521 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:18:21.926521 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:18:21.926521 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:18:21.926521 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:18:21.927505 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:21.927505 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:21.927505 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:18:21.927505 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:18:21.927505 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:18:21.927505 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:18:21.927505 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:21.927505 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:21.927505 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:21.927505 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:21.927505 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:21.927505 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:21.927505 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:21.927505 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:21.927505 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:21.927505 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:21.927505 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:21.927505 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:21.928455 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:21.928455 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:21.928455 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:18:21.928455 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:18:21.928455 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:18:21.928455 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:18:21.928455 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:18:21.928455 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:18:21.928455 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 01 17:18:21.928455 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:18:21.928455 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 285/590] 10.222.0.102 () {56 vars in 1107 bytes} [Thu Sep 1 17:18:21 2022] GET /identity/v3/users/neutron => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:21.929646 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5915418-3242-4dae-87d1-967c94dc813b None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:21.999133 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3971da3a-0f0f-4f1a-9fab-e3c97f403003 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:22.011727 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:22.048228 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04de843-81b8-4f86-8019-15ecd40978f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:22.049163 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04de843-81b8-4f86-8019-15ecd40978f8 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:22.049600 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04de843-81b8-4f86-8019-15ecd40978f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:22.049879 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04de843-81b8-4f86-8019-15ecd40978f8 None admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:22.051084 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04de843-81b8-4f86-8019-15ecd40978f8 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:22.051397 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04de843-81b8-4f86-8019-15ecd40978f8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:22.053418 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04de843-81b8-4f86-8019-15ecd40978f8 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:22.058063 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5915418-3242-4dae-87d1-967c94dc813b None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:22.075276 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 306/591] 10.222.0.102 () {56 vars in 1142 bytes} [Thu Sep 1 17:18:21 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 374 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:22.076546 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04de843-81b8-4f86-8019-15ecd40978f8 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:22.106009 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42d0fae-d708-444b-b2db-8b63cc330b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:22.107326 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42d0fae-d708-444b-b2db-8b63cc330b44 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:22.107977 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42d0fae-d708-444b-b2db-8b63cc330b44 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:22.108771 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42d0fae-d708-444b-b2db-8b63cc330b44 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:22.125300 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42d0fae-d708-444b-b2db-8b63cc330b44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:22.126788 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42d0fae-d708-444b-b2db-8b63cc330b44 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:22.142307 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-b42d0fae-d708-444b-b2db-8b63cc330b44 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:22.142307 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:22.142307 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:22.142307 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:22.142307 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:22.142307 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:22.142307 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:22.142307 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:22.142307 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:22.142307 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:22.142307 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:22.142307 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:22.142307 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:18:22.142307 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:18:22.142307 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:18:22.142307 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:18:22.143266 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:22.143266 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:22.143266 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:22.143266 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:22.143266 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:22.143266 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:22.143266 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:22.143266 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:22.143266 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:22.143266 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:22.143266 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:22.143266 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:22.143266 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:22.143266 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:22.143266 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:22.143266 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:22.143266 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:18:22.144160 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:18:22.144160 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:22.144160 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:18:22.144160 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 286/592] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:18:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:22.144880 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42d0fae-d708-444b-b2db-8b63cc330b44 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:22.200146 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04de843-81b8-4f86-8019-15ecd40978f8 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:22.213813 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:22.242488 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b2b740-0f63-4b36-8314-1cd36167979a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:22.243580 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b2b740-0f63-4b36-8314-1cd36167979a None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:22.244204 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b2b740-0f63-4b36-8314-1cd36167979a None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:22.244683 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b2b740-0f63-4b36-8314-1cd36167979a None admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:22.245778 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b2b740-0f63-4b36-8314-1cd36167979a None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:22.246341 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b2b740-0f63-4b36-8314-1cd36167979a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:22.247990 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b2b740-0f63-4b36-8314-1cd36167979a None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:22.251896 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42d0fae-d708-444b-b2db-8b63cc330b44 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:22.268486 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 307/593] 10.222.0.102 () {56 vars in 1144 bytes} [Thu Sep 1 17:18:22 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 376 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:22.269798 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b2b740-0f63-4b36-8314-1cd36167979a None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:22.278856 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:22.299349 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d569e3b5-3a33-4b39-8c89-7b8e0d14860f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:22.300612 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d569e3b5-3a33-4b39-8c89-7b8e0d14860f None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:22.301851 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d569e3b5-3a33-4b39-8c89-7b8e0d14860f None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:22.302630 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d569e3b5-3a33-4b39-8c89-7b8e0d14860f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:22.317830 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d569e3b5-3a33-4b39-8c89-7b8e0d14860f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:22.320038 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d569e3b5-3a33-4b39-8c89-7b8e0d14860f None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:22.341221 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-d569e3b5-3a33-4b39-8c89-7b8e0d14860f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:22.341221 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:22.341221 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:22.341221 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:22.341221 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:22.341221 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:22.341221 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:22.341221 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:22.341221 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:22.341221 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:22.341221 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:22.341221 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:22.341221 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:18:22.341221 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:18:22.341221 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:18:22.341221 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:18:22.342378 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:22.342378 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:22.342378 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:22.342378 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:22.342378 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:22.342378 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:22.342378 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:22.342378 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:22.342378 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:22.342378 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:22.342378 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:22.342378 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:22.342378 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:22.342378 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:22.342378 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:22.342378 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:22.342378 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:18:22.343283 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:18:22.343283 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:22.343283 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:18:22.346081 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 287/594] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:18:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:22.347249 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d569e3b5-3a33-4b39-8c89-7b8e0d14860f None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:22.395905 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b2b740-0f63-4b36-8314-1cd36167979a None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:22.433517 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2888d932-a65a-47f2-820f-071cb1a3ac60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:22.434129 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2888d932-a65a-47f2-820f-071cb1a3ac60 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:22.434409 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2888d932-a65a-47f2-820f-071cb1a3ac60 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:22.434747 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2888d932-a65a-47f2-820f-071cb1a3ac60 None admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:22.435398 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2888d932-a65a-47f2-820f-071cb1a3ac60 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:22.435701 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2888d932-a65a-47f2-820f-071cb1a3ac60 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:22.437084 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2888d932-a65a-47f2-820f-071cb1a3ac60 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:22.453624 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 308/595] 10.222.0.102 () {56 vars in 1114 bytes} [Thu Sep 1 17:18:22 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:22.454555 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2888d932-a65a-47f2-820f-071cb1a3ac60 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:22.465099 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d569e3b5-3a33-4b39-8c89-7b8e0d14860f None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:22.465835 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:22.514680 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601c2a65-a0c6-422c-b49e-a0766e371bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:22.515709 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601c2a65-a0c6-422c-b49e-a0766e371bcf None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:22.516195 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601c2a65-a0c6-422c-b49e-a0766e371bcf None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:22.516608 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601c2a65-a0c6-422c-b49e-a0766e371bcf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:22.529115 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601c2a65-a0c6-422c-b49e-a0766e371bcf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:22.530923 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601c2a65-a0c6-422c-b49e-a0766e371bcf None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:22.546995 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-601c2a65-a0c6-422c-b49e-a0766e371bcf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:22.546995 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:22.546995 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:22.546995 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:22.546995 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:22.546995 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:22.546995 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:22.546995 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:22.546995 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:22.546995 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:22.546995 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:22.546995 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:22.546995 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:18:22.546995 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:18:22.546995 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:18:22.546995 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:18:22.548150 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:22.548150 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:22.548150 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:22.548150 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:22.548150 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:22.548150 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:22.548150 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:22.548150 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:22.548150 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:22.548150 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:22.548150 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:22.548150 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:22.548150 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:22.548150 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:22.548150 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:22.548150 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:22.548150 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:18:22.549071 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:18:22.549071 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:22.549071 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:18:22.550563 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 288/596] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:18:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:22.552051 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601c2a65-a0c6-422c-b49e-a0766e371bcf None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:22.583876 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:22.585045 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2888d932-a65a-47f2-820f-071cb1a3ac60 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:22.623241 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afb7355-8c3a-43ad-bb0f-c3812e09ab7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:22.623840 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afb7355-8c3a-43ad-bb0f-c3812e09ab7a None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:22.624125 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afb7355-8c3a-43ad-bb0f-c3812e09ab7a None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:22.624399 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afb7355-8c3a-43ad-bb0f-c3812e09ab7a None admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:22.625038 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afb7355-8c3a-43ad-bb0f-c3812e09ab7a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:22.625392 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afb7355-8c3a-43ad-bb0f-c3812e09ab7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:22.626737 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afb7355-8c3a-43ad-bb0f-c3812e09ab7a None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:22.644379 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 309/597] 10.222.0.102 () {56 vars in 1114 bytes} [Thu Sep 1 17:18:22 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:22.644379 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afb7355-8c3a-43ad-bb0f-c3812e09ab7a None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:22.672810 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601c2a65-a0c6-422c-b49e-a0766e371bcf None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:22.709723 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45581bde-24ac-40eb-bafe-4ec843150394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:22.710828 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45581bde-24ac-40eb-bafe-4ec843150394 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:22.710828 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45581bde-24ac-40eb-bafe-4ec843150394 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:22.711011 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45581bde-24ac-40eb-bafe-4ec843150394 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:22.711510 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:22.718305 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45581bde-24ac-40eb-bafe-4ec843150394 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:22.719755 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45581bde-24ac-40eb-bafe-4ec843150394 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:22.731577 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-45581bde-24ac-40eb-bafe-4ec843150394 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:18:22.731577 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:22.731577 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:22.731577 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:22.731577 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:22.731577 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:22.731577 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:22.731577 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:22.731577 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:22.731577 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:22.731577 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:22.731577 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:22.731577 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:18:22.731577 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:18:22.731577 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:18:22.732540 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:18:22.732540 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:22.732540 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:22.732540 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:22.732540 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:22.732540 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:22.732540 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:22.732540 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:22.732540 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:22.732540 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:22.732540 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:22.732540 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:22.732540 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:22.732540 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:22.732540 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:22.732540 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:22.732540 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:22.733813 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:18:22.733813 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:18:22.733813 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:18:22.733813 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:18:22.733813 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:18:22.733813 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:18:22.733813 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:18:22.733813 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:18:22.733813 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 289/598] 10.222.0.102 () {56 vars in 1116 bytes} [Thu Sep 1 17:18:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:22.733813 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1433e6-302d-4d7b-8e79-7c0b41cbf7be None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:22.734347 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1433e6-302d-4d7b-8e79-7c0b41cbf7be None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:22.734347 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1433e6-302d-4d7b-8e79-7c0b41cbf7be None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:22.735583 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 290/599] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:18:22 2022] GET /identity => generated 271 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:18:22.736188 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1433e6-302d-4d7b-8e79-7c0b41cbf7be None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:22.758166 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afb7355-8c3a-43ad-bb0f-c3812e09ab7a None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:22.793748 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a73ced-73d1-42df-9361-ae089e36c949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:22.794412 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a73ced-73d1-42df-9361-ae089e36c949 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:22.794694 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a73ced-73d1-42df-9361-ae089e36c949 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:22.794970 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a73ced-73d1-42df-9361-ae089e36c949 None admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:22.795608 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a73ced-73d1-42df-9361-ae089e36c949 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:22.795912 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a73ced-73d1-42df-9361-ae089e36c949 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:22.797274 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a73ced-73d1-42df-9361-ae089e36c949 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:22.810477 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 310/600] 10.222.0.102 () {56 vars in 1114 bytes} [Thu Sep 1 17:18:22 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:22.811709 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a73ced-73d1-42df-9361-ae089e36c949 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:22.852563 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:22.874471 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1433e6-302d-4d7b-8e79-7c0b41cbf7be None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:22.913139 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444f2251-b6a7-402b-9090-916a306df71c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:22.913811 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444f2251-b6a7-402b-9090-916a306df71c None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:22.914098 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444f2251-b6a7-402b-9090-916a306df71c None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:22.914375 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444f2251-b6a7-402b-9090-916a306df71c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:22.921488 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444f2251-b6a7-402b-9090-916a306df71c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:22.922892 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a73ced-73d1-42df-9361-ae089e36c949 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:22.923025 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444f2251-b6a7-402b-9090-916a306df71c None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:22.935040 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-444f2251-b6a7-402b-9090-916a306df71c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:18:22.935040 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:22.935040 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:22.935040 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:22.935040 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:22.935040 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:22.935040 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:22.935040 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:22.935040 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:22.935040 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:22.935040 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:22.935040 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:22.935040 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:18:22.935040 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:18:22.935040 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:18:22.936101 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:18:22.936101 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:22.936101 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:22.936101 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:22.936101 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:22.936101 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:22.936101 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:22.936101 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:22.936101 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:22.936101 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:22.936101 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:22.936101 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:22.936101 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:22.936101 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:22.936101 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:22.936101 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:22.936101 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:22.937116 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:18:22.937116 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:18:22.937116 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:18:22.937116 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:18:22.937116 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:18:22.937116 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:18:22.937116 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:18:22.937116 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:18:22.937116 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 291/601] 10.222.0.102 () {56 vars in 1116 bytes} [Thu Sep 1 17:18:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:22.937634 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36db6420-16ae-4304-bc72-0b1f814e2516 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:22.937920 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36db6420-16ae-4304-bc72-0b1f814e2516 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:22.938198 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36db6420-16ae-4304-bc72-0b1f814e2516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:22.958106 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-36db6420-16ae-4304-bc72-0b1f814e2516 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:18:22.959337 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-36db6420-16ae-4304-bc72-0b1f814e2516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:18:22.960351 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:22.969642 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf5403e-6854-430e-a744-914352361960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:22.970250 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf5403e-6854-430e-a744-914352361960 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:22.970530 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf5403e-6854-430e-a744-914352361960 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:22.970797 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf5403e-6854-430e-a744-914352361960 None admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:22.971485 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf5403e-6854-430e-a744-914352361960 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:22.971792 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf5403e-6854-430e-a744-914352361960 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:22.973759 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf5403e-6854-430e-a744-914352361960 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:22.992409 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 311/602] 10.222.0.102 () {56 vars in 1153 bytes} [Thu Sep 1 17:18:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:22.992409 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf5403e-6854-430e-a744-914352361960 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:23.037904 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 292/603] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:18:22 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:18:23.042368 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36db6420-16ae-4304-bc72-0b1f814e2516 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:23.112176 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf5403e-6854-430e-a744-914352361960 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:23.139051 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:23.154357 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0199f9fc-e1f2-4889-a5a4-23ca7f6a925d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:23.155449 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0199f9fc-e1f2-4889-a5a4-23ca7f6a925d None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:23.155925 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0199f9fc-e1f2-4889-a5a4-23ca7f6a925d None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:23.156393 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0199f9fc-e1f2-4889-a5a4-23ca7f6a925d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:23.166937 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0199f9fc-e1f2-4889-a5a4-23ca7f6a925d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:23.168952 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0199f9fc-e1f2-4889-a5a4-23ca7f6a925d None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:23.177568 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36db6420-16ae-4304-bc72-0b1f814e2516 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:23.183190 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-0199f9fc-e1f2-4889-a5a4-23ca7f6a925d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:18:23.183190 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:23.183190 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:23.183190 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:23.183190 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:23.183190 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:23.183190 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:23.183190 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:23.183190 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:23.183190 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:23.183190 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:23.183190 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:23.183190 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:18:23.183190 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:18:23.183190 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:18:23.184042 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:18:23.184042 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:23.184042 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:23.184042 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:23.184042 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:23.184042 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:23.184042 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:23.184042 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:23.184042 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:23.184042 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:23.184042 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:23.184042 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:23.184042 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:23.184042 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:23.184042 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:23.184042 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:23.184042 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:23.184916 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:18:23.184916 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:18:23.184916 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:18:23.184916 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:18:23.184916 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:18:23.184916 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:18:23.184916 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:18:23.184916 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:18:23.187559 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 312/604] 10.222.0.102 () {56 vars in 1116 bytes} [Thu Sep 1 17:18:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:23.188672 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0199f9fc-e1f2-4889-a5a4-23ca7f6a925d None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:23.217400 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6b571f-2ce6-4c19-be13-47aad539c270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:23.218019 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6b571f-2ce6-4c19-be13-47aad539c270 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:23.218307 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6b571f-2ce6-4c19-be13-47aad539c270 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:23.218783 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6b571f-2ce6-4c19-be13-47aad539c270 None admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:23.219628 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6b571f-2ce6-4c19-be13-47aad539c270 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:23.223194 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6b571f-2ce6-4c19-be13-47aad539c270 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:23.223194 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6b571f-2ce6-4c19-be13-47aad539c270 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:23.241478 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 293/605] 10.222.0.102 () {56 vars in 1153 bytes} [Thu Sep 1 17:18:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:23.243257 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6b571f-2ce6-4c19-be13-47aad539c270 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:23.273236 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:23.338860 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0199f9fc-e1f2-4889-a5a4-23ca7f6a925d None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:23.355776 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6b571f-2ce6-4c19-be13-47aad539c270 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:23.379017 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e70c74c-cb4e-4a9f-82dc-016af23d7998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:23.379918 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e70c74c-cb4e-4a9f-82dc-016af23d7998 None admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:23.380264 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e70c74c-cb4e-4a9f-82dc-016af23d7998 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:23.380891 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e70c74c-cb4e-4a9f-82dc-016af23d7998 None admin] PATH_INFO: `/v3/projects/e78fff7f1f6e4f68b8d233ad2e5bbd4f/users/039683b8d1d146c6add5ffa235d06f91/roles/f21e23654e4740158c7d4f46e726e515` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:23.391022 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca29379-4f91-47f1-ac95-2e02eab5250b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:23.391634 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca29379-4f91-47f1-ac95-2e02eab5250b None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:23.391916 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca29379-4f91-47f1-ac95-2e02eab5250b None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:23.392188 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca29379-4f91-47f1-ac95-2e02eab5250b None admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:23.392875 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca29379-4f91-47f1-ac95-2e02eab5250b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:23.393185 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca29379-4f91-47f1-ac95-2e02eab5250b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:23.395180 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca29379-4f91-47f1-ac95-2e02eab5250b None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:23.396950 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:23.397060 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e70c74c-cb4e-4a9f-82dc-016af23d7998 None admin] RBAC: Authorizing `identity:create_grant(project_id=e78fff7f1f6e4f68b8d233ad2e5bbd4f, user_id=039683b8d1d146c6add5ffa235d06f91, role_id=f21e23654e4740158c7d4f46e726e515)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:23.399080 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e70c74c-cb4e-4a9f-82dc-016af23d7998 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:23.414847 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 313/606] 10.222.0.102 () {58 vars in 1444 bytes} [Thu Sep 1 17:18:23 2022] PUT /identity/v3/projects/e78fff7f1f6e4f68b8d233ad2e5bbd4f/users/039683b8d1d146c6add5ffa235d06f91/roles/f21e23654e4740158c7d4f46e726e515 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:18:23.414847 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e70c74c-cb4e-4a9f-82dc-016af23d7998 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:23.421751 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 294/607] 10.222.0.102 () {56 vars in 1153 bytes} [Thu Sep 1 17:18:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:23.423084 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca29379-4f91-47f1-ac95-2e02eab5250b None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:23.552165 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:23.557199 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca29379-4f91-47f1-ac95-2e02eab5250b None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:23.559195 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e70c74c-cb4e-4a9f-82dc-016af23d7998 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:23.610997 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ce6fe1-17b0-41cb-86d4-d3cff0508248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:23.611641 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ce6fe1-17b0-41cb-86d4-d3cff0508248 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:23.611923 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ce6fe1-17b0-41cb-86d4-d3cff0508248 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:23.612050 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1e81fb-e341-4157-9ad3-6c34781493a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:23.612237 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ce6fe1-17b0-41cb-86d4-d3cff0508248 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:23.612738 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1e81fb-e341-4157-9ad3-6c34781493a1 None admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:23.613071 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1e81fb-e341-4157-9ad3-6c34781493a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:23.613425 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1e81fb-e341-4157-9ad3-6c34781493a1 None admin] PATH_INFO: `/v3/projects/e78fff7f1f6e4f68b8d233ad2e5bbd4f/users/523ed589872a45a09b6a7675a7fb0b75/roles/f21e23654e4740158c7d4f46e726e515` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:23.623669 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ce6fe1-17b0-41cb-86d4-d3cff0508248 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:18:23.628654 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-56ce6fe1-17b0-41cb-86d4-d3cff0508248 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:18:23.628654 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:23.628654 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:23.628654 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:23.628654 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:23.628654 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:23.628654 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:23.628654 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:23.628654 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:23.628654 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:23.628654 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:23.628654 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:23.628654 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:18:23.628654 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:18:23.628654 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:18:23.628654 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:18:23.630299 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:18:23.630299 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:18:23.630299 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:18:23.630299 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:18:23.630299 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:23.630299 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:23.630299 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:23.630299 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:23.630299 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:23.630299 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:23.630299 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:23.630299 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:23.630299 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:23.630299 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:23.630299 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:23.630299 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:23.631422 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:23.631422 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:23.631422 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:23.631422 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:23.631422 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:18:23.631422 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:18:23.631422 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:18:23.631422 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:18:23.631422 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:18:23.631422 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:18:23.631422 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:18:23.631422 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:18:23.632166 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1e81fb-e341-4157-9ad3-6c34781493a1 None admin] RBAC: Authorizing `identity:create_grant(project_id=e78fff7f1f6e4f68b8d233ad2e5bbd4f, user_id=523ed589872a45a09b6a7675a7fb0b75, role_id=f21e23654e4740158c7d4f46e726e515)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:23.632422 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1e81fb-e341-4157-9ad3-6c34781493a1 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:23.634154 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 295/608] 10.222.0.102 () {56 vars in 1107 bytes} [Thu Sep 1 17:18:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:23.635225 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ce6fe1-17b0-41cb-86d4-d3cff0508248 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:23.652025 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 314/609] 10.222.0.102 () {58 vars in 1444 bytes} [Thu Sep 1 17:18:23 2022] PUT /identity/v3/projects/e78fff7f1f6e4f68b8d233ad2e5bbd4f/users/523ed589872a45a09b6a7675a7fb0b75/roles/f21e23654e4740158c7d4f46e726e515 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:18:23.653677 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1e81fb-e341-4157-9ad3-6c34781493a1 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:23.757181 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:23.812613 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ce6fe1-17b0-41cb-86d4-d3cff0508248 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:23.852676 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6241195a-265e-4acc-ad1c-e59dd7b2a82a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:23.853442 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6241195a-265e-4acc-ad1c-e59dd7b2a82a None admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:23.854910 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6241195a-265e-4acc-ad1c-e59dd7b2a82a None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:23.854910 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6241195a-265e-4acc-ad1c-e59dd7b2a82a None admin] PATH_INFO: `/v3/projects/e78fff7f1f6e4f68b8d233ad2e5bbd4f/users/7c7d3594fdab4191be1d841af43a5c79/roles/f21e23654e4740158c7d4f46e726e515` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:23.867772 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1e81fb-e341-4157-9ad3-6c34781493a1 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:23.877412 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6241195a-265e-4acc-ad1c-e59dd7b2a82a None admin] RBAC: Authorizing `identity:create_grant(project_id=e78fff7f1f6e4f68b8d233ad2e5bbd4f, user_id=7c7d3594fdab4191be1d841af43a5c79, role_id=f21e23654e4740158c7d4f46e726e515)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:23.879571 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6241195a-265e-4acc-ad1c-e59dd7b2a82a None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:23.896761 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 296/610] 10.222.0.102 () {58 vars in 1444 bytes} [Thu Sep 1 17:18:23 2022] PUT /identity/v3/projects/e78fff7f1f6e4f68b8d233ad2e5bbd4f/users/7c7d3594fdab4191be1d841af43a5c79/roles/f21e23654e4740158c7d4f46e726e515 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:18:23.898875 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38171ab9-33b6-4fb8-a365-aeb91a3d90ed None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:23.898875 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38171ab9-33b6-4fb8-a365-aeb91a3d90ed None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:23.898875 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38171ab9-33b6-4fb8-a365-aeb91a3d90ed None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:23.900518 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 297/611] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:18:23 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:18:23.908295 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20e68b5-a667-4794-b26b-7b21001c7502 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:23.908715 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20e68b5-a667-4794-b26b-7b21001c7502 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:23.909088 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20e68b5-a667-4794-b26b-7b21001c7502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:23.919066 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b38078-ccb2-4cf4-96aa-b186c7abc69f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:23.920030 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b38078-ccb2-4cf4-96aa-b186c7abc69f None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:23.920461 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b38078-ccb2-4cf4-96aa-b186c7abc69f None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:23.920882 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b38078-ccb2-4cf4-96aa-b186c7abc69f None admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:23.921807 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b38078-ccb2-4cf4-96aa-b186c7abc69f None admin] RBAC: Adding query filter params (name=service) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:23.922338 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b38078-ccb2-4cf4-96aa-b186c7abc69f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:23.924169 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b38078-ccb2-4cf4-96aa-b186c7abc69f None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:23.936326 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-b20e68b5-a667-4794-b26b-7b21001c7502 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:18:23.936886 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 315/612] 10.222.0.102 () {56 vars in 1108 bytes} [Thu Sep 1 17:18:23 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:23.941935 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-b20e68b5-a667-4794-b26b-7b21001c7502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:18:23.946687 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b38078-ccb2-4cf4-96aa-b186c7abc69f None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:23.985720 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:24.079918 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 298/613] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:18:23 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:18:24.108848 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b38078-ccb2-4cf4-96aa-b186c7abc69f None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:24.149396 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8fef3d-e27e-497b-bf31-6b5ed3604c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:24.150019 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8fef3d-e27e-497b-bf31-6b5ed3604c38 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:24.150301 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8fef3d-e27e-497b-bf31-6b5ed3604c38 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:24.150573 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8fef3d-e27e-497b-bf31-6b5ed3604c38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:24.161884 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8fef3d-e27e-497b-bf31-6b5ed3604c38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:24.163309 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8fef3d-e27e-497b-bf31-6b5ed3604c38 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:24.178829 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-fb8fef3d-e27e-497b-bf31-6b5ed3604c38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:24.178829 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:24.178829 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:24.178829 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:24.178829 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:24.178829 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:24.178829 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:24.178829 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:24.178829 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:24.178829 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:24.178829 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:24.178829 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:24.178829 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:18:24.178829 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:18:24.178829 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:18:24.178829 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:18:24.179822 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:24.179822 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:24.179822 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:24.179822 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:24.179822 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:24.179822 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:24.179822 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:24.179822 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:24.179822 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:24.179822 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:24.179822 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:24.179822 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:24.179822 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:24.179822 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:24.179822 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:24.179822 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:24.179822 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:18:24.180700 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:18:24.180700 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:24.180700 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:18:24.180700 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 316/614] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:18:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:24.187688 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20e68b5-a667-4794-b26b-7b21001c7502 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:24.240982 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:24.318696 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20e68b5-a667-4794-b26b-7b21001c7502 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:24.375925 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adda503c-27b9-4e08-9cfa-55323db48fc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:24.376515 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adda503c-27b9-4e08-9cfa-55323db48fc4 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:24.376707 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adda503c-27b9-4e08-9cfa-55323db48fc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:24.376977 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adda503c-27b9-4e08-9cfa-55323db48fc4 None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:24.377645 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adda503c-27b9-4e08-9cfa-55323db48fc4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:24.377949 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adda503c-27b9-4e08-9cfa-55323db48fc4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:24.379980 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adda503c-27b9-4e08-9cfa-55323db48fc4 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:24.396243 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 299/615] 10.222.0.102 () {56 vars in 1114 bytes} [Thu Sep 1 17:18:24 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:24.401717 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8fef3d-e27e-497b-bf31-6b5ed3604c38 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:24.496329 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adda503c-27b9-4e08-9cfa-55323db48fc4 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:24.512570 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:24.558289 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8fef3d-e27e-497b-bf31-6b5ed3604c38 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:24.594790 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4900ee0f-2279-4699-b107-474b63a22e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:24.595752 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4900ee0f-2279-4699-b107-474b63a22e2d None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:24.596211 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4900ee0f-2279-4699-b107-474b63a22e2d None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:24.596659 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4900ee0f-2279-4699-b107-474b63a22e2d None admin] PATH_INFO: `/v3/users/swift` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:24.605777 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4900ee0f-2279-4699-b107-474b63a22e2d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:24.607557 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4900ee0f-2279-4699-b107-474b63a22e2d None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:24.619924 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-4900ee0f-2279-4699-b107-474b63a22e2d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 01 17:18:24.619924 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:24.619924 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:24.619924 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:24.619924 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:24.619924 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:24.619924 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:24.619924 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:24.619924 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:24.619924 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:24.619924 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:24.619924 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:24.619924 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:18:24.619924 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:18:24.619924 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:18:24.619924 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:18:24.620898 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:24.620898 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:24.620898 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:18:24.620898 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:18:24.620898 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:18:24.620898 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:18:24.620898 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:24.620898 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:24.620898 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:24.620898 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:24.620898 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:24.620898 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:24.620898 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:24.620898 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:24.620898 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:24.620898 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:24.620898 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:24.620898 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:24.621873 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:24.621873 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:24.621873 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:18:24.621873 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:18:24.621873 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:18:24.621873 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:18:24.621873 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:18:24.621873 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:18:24.621873 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 01 17:18:24.621873 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:18:24.624461 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 317/616] 10.222.0.102 () {56 vars in 1101 bytes} [Thu Sep 1 17:18:24 2022] GET /identity/v3/users/swift => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:24.630259 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4900ee0f-2279-4699-b107-474b63a22e2d None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:24.683637 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:24.696531 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adda503c-27b9-4e08-9cfa-55323db48fc4 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:24.762319 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d8f3a2-b9c3-420a-a7d8-45c963502ecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:24.763365 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d8f3a2-b9c3-420a-a7d8-45c963502ecd None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:24.763842 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d8f3a2-b9c3-420a-a7d8-45c963502ecd None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:24.764309 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d8f3a2-b9c3-420a-a7d8-45c963502ecd None admin] PATH_INFO: `/v3/roles/service` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:24.782598 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4900ee0f-2279-4699-b107-474b63a22e2d None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:24.784698 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d8f3a2-b9c3-420a-a7d8-45c963502ecd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:18:24.791557 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-27d8f3a2-b9c3-420a-a7d8-45c963502ecd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:18:24.791557 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:24.791557 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:24.791557 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:24.791557 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:24.791557 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:24.791557 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:24.791557 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:24.791557 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:24.791557 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:24.791557 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:24.791557 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:24.791557 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:18:24.791557 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:18:24.791557 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:18:24.791557 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:18:24.792474 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:18:24.792474 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:18:24.792474 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:18:24.792474 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:18:24.792474 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:24.792474 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:24.792474 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:24.792474 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:24.792474 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:24.792474 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:24.792474 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:24.792474 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:24.792474 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:24.792474 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:24.792474 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:24.792474 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:24.793280 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:24.793280 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:24.793280 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:24.793280 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:24.793280 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:18:24.793280 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:18:24.793280 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:18:24.793280 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:18:24.793280 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:18:24.793280 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:18:24.793280 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:18:24.793280 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:18:24.806219 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 300/617] 10.222.0.102 () {56 vars in 1107 bytes} [Thu Sep 1 17:18:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:18:24.809790 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d8f3a2-b9c3-420a-a7d8-45c963502ecd None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:24.858370 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9a45b2-ba5f-4a05-8e8e-ea70c87da81a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:24.859330 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9a45b2-ba5f-4a05-8e8e-ea70c87da81a None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:24.859809 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9a45b2-ba5f-4a05-8e8e-ea70c87da81a None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:24.860278 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9a45b2-ba5f-4a05-8e8e-ea70c87da81a None admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:24.861297 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9a45b2-ba5f-4a05-8e8e-ea70c87da81a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:24.861827 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9a45b2-ba5f-4a05-8e8e-ea70c87da81a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:24.863497 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9a45b2-ba5f-4a05-8e8e-ea70c87da81a None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:24.877541 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:24.898983 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 318/618] 10.222.0.102 () {56 vars in 1140 bytes} [Thu Sep 1 17:18:24 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 372 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:18:24.904704 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9a45b2-ba5f-4a05-8e8e-ea70c87da81a None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:24.991532 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d8f3a2-b9c3-420a-a7d8-45c963502ecd None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:25.079438 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db769b32-7345-4105-ab75-e88a426c5347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:25.080052 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db769b32-7345-4105-ab75-e88a426c5347 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:25.080332 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db769b32-7345-4105-ab75-e88a426c5347 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:25.080602 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db769b32-7345-4105-ab75-e88a426c5347 None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:25.081266 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db769b32-7345-4105-ab75-e88a426c5347 None admin] RBAC: Adding query filter params (name=service) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:25.081592 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db769b32-7345-4105-ab75-e88a426c5347 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:25.083496 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db769b32-7345-4105-ab75-e88a426c5347 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:25.101451 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9a45b2-ba5f-4a05-8e8e-ea70c87da81a None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:25.105510 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 301/619] 10.222.0.102 () {56 vars in 1108 bytes} [Thu Sep 1 17:18:24 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:25.110894 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db769b32-7345-4105-ab75-e88a426c5347 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:25.153757 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:25.187019 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78ac388-76c1-4ae5-9e08-888b928ac1fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:25.188376 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78ac388-76c1-4ae5-9e08-888b928ac1fe None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:25.189020 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78ac388-76c1-4ae5-9e08-888b928ac1fe None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:25.189514 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78ac388-76c1-4ae5-9e08-888b928ac1fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:25.211388 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78ac388-76c1-4ae5-9e08-888b928ac1fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:25.213163 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78ac388-76c1-4ae5-9e08-888b928ac1fe None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:25.246335 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db769b32-7345-4105-ab75-e88a426c5347 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:25.246710 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-a78ac388-76c1-4ae5-9e08-888b928ac1fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:25.246710 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:25.246710 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:25.246710 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:25.246710 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:25.246710 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:25.246710 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:25.246710 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:25.246710 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:25.246710 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:25.246710 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:25.246710 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:25.246710 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:18:25.246710 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:18:25.246710 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:18:25.246710 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:18:25.247609 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:25.247609 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:25.247609 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:25.247609 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:25.247609 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:25.247609 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:25.247609 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:25.247609 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:25.247609 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:25.247609 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:25.247609 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:25.247609 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:25.247609 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:25.247609 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:25.247609 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:25.247609 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:25.247609 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:18:25.248474 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:18:25.248474 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:25.248474 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:18:25.250704 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 319/620] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:18:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:25.256502 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78ac388-76c1-4ae5-9e08-888b928ac1fe None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:25.282758 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da7bb8b-79c0-44e9-ad0b-2c72ee007917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:25.283407 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da7bb8b-79c0-44e9-ad0b-2c72ee007917 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:25.283689 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da7bb8b-79c0-44e9-ad0b-2c72ee007917 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:25.283965 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da7bb8b-79c0-44e9-ad0b-2c72ee007917 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:25.298095 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da7bb8b-79c0-44e9-ad0b-2c72ee007917 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:25.299769 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da7bb8b-79c0-44e9-ad0b-2c72ee007917 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:25.316912 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-4da7bb8b-79c0-44e9-ad0b-2c72ee007917 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:25.316912 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:25.316912 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:25.316912 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:25.316912 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:25.316912 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:25.316912 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:25.316912 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:25.316912 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:25.316912 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:25.316912 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:25.316912 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:25.316912 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:18:25.316912 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:18:25.316912 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:18:25.316912 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:18:25.317901 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:25.317901 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:25.317901 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:25.317901 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:25.317901 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:25.317901 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:25.317901 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:25.317901 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:25.317901 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:25.317901 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:25.317901 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:25.317901 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:25.317901 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:25.317901 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:25.317901 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:25.317901 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:25.317901 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:18:25.318766 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:18:25.318766 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:25.318766 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:18:25.318766 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 302/621] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:18:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:18:25.325049 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da7bb8b-79c0-44e9-ad0b-2c72ee007917 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:25.338366 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:25.402859 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:25.440760 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da7bb8b-79c0-44e9-ad0b-2c72ee007917 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:25.481428 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecfa5a8-e90c-479c-a8db-6d66cdd4ab33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:25.482168 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecfa5a8-e90c-479c-a8db-6d66cdd4ab33 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:25.482458 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecfa5a8-e90c-479c-a8db-6d66cdd4ab33 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:25.482730 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecfa5a8-e90c-479c-a8db-6d66cdd4ab33 None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:25.483534 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecfa5a8-e90c-479c-a8db-6d66cdd4ab33 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:25.483848 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecfa5a8-e90c-479c-a8db-6d66cdd4ab33 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:25.485556 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecfa5a8-e90c-479c-a8db-6d66cdd4ab33 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:25.499224 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 303/622] 10.222.0.102 () {56 vars in 1114 bytes} [Thu Sep 1 17:18:25 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:25.505634 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecfa5a8-e90c-479c-a8db-6d66cdd4ab33 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:25.527325 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78ac388-76c1-4ae5-9e08-888b928ac1fe None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:25.587489 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb0c962-c5f1-4e23-8cfb-bd786e019566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:25.588236 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb0c962-c5f1-4e23-8cfb-bd786e019566 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:25.588519 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb0c962-c5f1-4e23-8cfb-bd786e019566 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:25.588793 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb0c962-c5f1-4e23-8cfb-bd786e019566 None admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:25.589469 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb0c962-c5f1-4e23-8cfb-bd786e019566 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:25.589775 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb0c962-c5f1-4e23-8cfb-bd786e019566 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:25.591135 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb0c962-c5f1-4e23-8cfb-bd786e019566 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:25.613852 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 320/623] 10.222.0.102 () {56 vars in 1114 bytes} [Thu Sep 1 17:18:25 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:25.614241 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecfa5a8-e90c-479c-a8db-6d66cdd4ab33 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:25.615251 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce69aaf3-75dd-4f38-acbe-880fd39dbd8d None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:25.615539 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce69aaf3-75dd-4f38-acbe-880fd39dbd8d None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:25.615814 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce69aaf3-75dd-4f38-acbe-880fd39dbd8d None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:25.617500 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 321/624] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:18:25 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:18:25.622347 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce69aaf3-75dd-4f38-acbe-880fd39dbd8d None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:25.652224 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:25.653879 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f815701f-33ab-4235-a7bd-4682583430ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:25.654974 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f815701f-33ab-4235-a7bd-4682583430ba None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:25.655834 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f815701f-33ab-4235-a7bd-4682583430ba None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:25.656913 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f815701f-33ab-4235-a7bd-4682583430ba None admin] PATH_INFO: `/v3/users/nova` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:25.668349 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f815701f-33ab-4235-a7bd-4682583430ba None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:25.670998 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f815701f-33ab-4235-a7bd-4682583430ba None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:25.686068 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-f815701f-33ab-4235-a7bd-4682583430ba None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 17:18:25.686068 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:25.686068 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:25.686068 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:25.686068 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:25.686068 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:25.686068 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:25.686068 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:25.686068 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:25.686068 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:25.686068 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:25.686068 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:25.686068 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:18:25.686068 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:18:25.686068 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:18:25.686068 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:18:25.687106 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:25.687106 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:25.687106 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:18:25.687106 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:18:25.687106 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:18:25.687106 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:18:25.687106 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:25.687106 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:25.687106 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:25.687106 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:25.687106 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:25.687106 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:25.687106 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:25.687106 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:25.687106 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:25.687106 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:25.687106 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:25.687106 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:25.688038 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:25.688038 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:25.688038 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:18:25.688038 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:18:25.688038 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:18:25.688038 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:18:25.688038 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:18:25.688038 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:18:25.688038 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 17:18:25.688038 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:18:25.692582 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 304/625] 10.222.0.102 () {56 vars in 1098 bytes} [Thu Sep 1 17:18:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:18:25.693952 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f70ab4-41f5-4e89-ab7b-26a8dad1460b None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:25.694320 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f70ab4-41f5-4e89-ab7b-26a8dad1460b None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:25.694922 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f70ab4-41f5-4e89-ab7b-26a8dad1460b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:25.712641 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-80f70ab4-41f5-4e89-ab7b-26a8dad1460b None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:18:25.713984 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-80f70ab4-41f5-4e89-ab7b-26a8dad1460b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:18:25.741900 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:25.756040 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce69aaf3-75dd-4f38-acbe-880fd39dbd8d None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:25.785003 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 305/626] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:18:25 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:18:25.790188 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f70ab4-41f5-4e89-ab7b-26a8dad1460b None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:25.804290 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aa0abc-d363-4fd3-8376-cd776842cd0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:25.805340 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aa0abc-d363-4fd3-8376-cd776842cd0a None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:25.805899 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aa0abc-d363-4fd3-8376-cd776842cd0a None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:25.806819 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aa0abc-d363-4fd3-8376-cd776842cd0a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:25.819469 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18aa0abc-d363-4fd3-8376-cd776842cd0a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:25.821381 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18aa0abc-d363-4fd3-8376-cd776842cd0a None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:25.834926 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-18aa0abc-d363-4fd3-8376-cd776842cd0a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:18:25.834926 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:25.834926 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:25.834926 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:25.834926 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:25.834926 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:25.834926 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:25.834926 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:25.834926 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:25.834926 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:25.834926 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:25.834926 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:25.834926 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:18:25.834926 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:18:25.834926 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:18:25.835899 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:18:25.835899 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:25.835899 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:25.835899 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:25.835899 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:25.835899 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:25.835899 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:25.835899 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:25.835899 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:25.835899 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:25.835899 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:25.835899 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:25.835899 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:25.835899 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:25.835899 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:25.835899 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:25.835899 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:25.836778 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:18:25.836778 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:18:25.836778 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:18:25.836778 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:18:25.836778 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:18:25.836778 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:18:25.836778 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:18:25.836778 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:18:25.839431 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 322/627] 10.222.0.102 () {56 vars in 1116 bytes} [Thu Sep 1 17:18:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:25.849793 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aa0abc-d363-4fd3-8376-cd776842cd0a None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:25.979182 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aa0abc-d363-4fd3-8376-cd776842cd0a None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:26.020794 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d1f97c-2ed5-401f-a4a5-0f7d688bede7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:26.022084 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d1f97c-2ed5-401f-a4a5-0f7d688bede7 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:26.022572 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d1f97c-2ed5-401f-a4a5-0f7d688bede7 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:26.023052 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d1f97c-2ed5-401f-a4a5-0f7d688bede7 None admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:26.023942 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d1f97c-2ed5-401f-a4a5-0f7d688bede7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:26.024495 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d1f97c-2ed5-401f-a4a5-0f7d688bede7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:26.026657 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d1f97c-2ed5-401f-a4a5-0f7d688bede7 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:26.041493 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 323/628] 10.222.0.102 () {56 vars in 1153 bytes} [Thu Sep 1 17:18:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:26.043808 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f70ab4-41f5-4e89-ab7b-26a8dad1460b None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:26.047860 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d1f97c-2ed5-401f-a4a5-0f7d688bede7 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:26.065687 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:26.094712 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:26.127689 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cbff75-df60-4dfc-8c2b-a27c4906792d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:26.128305 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cbff75-df60-4dfc-8c2b-a27c4906792d None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:26.128586 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cbff75-df60-4dfc-8c2b-a27c4906792d None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:26.128890 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cbff75-df60-4dfc-8c2b-a27c4906792d None admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:26.129669 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cbff75-df60-4dfc-8c2b-a27c4906792d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:26.129974 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cbff75-df60-4dfc-8c2b-a27c4906792d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:26.131544 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cbff75-df60-4dfc-8c2b-a27c4906792d None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:26.152059 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 306/629] 10.222.0.102 () {56 vars in 1138 bytes} [Thu Sep 1 17:18:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 370 bytes in 366 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:26.153472 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02f1c6e-f865-4d0a-b973-4cb2e5fde8b4 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:26.153804 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02f1c6e-f865-4d0a-b973-4cb2e5fde8b4 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:26.154130 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02f1c6e-f865-4d0a-b973-4cb2e5fde8b4 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:26.159364 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 307/630] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:18:26 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:18:26.160852 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02f1c6e-f865-4d0a-b973-4cb2e5fde8b4 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:26.172570 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d1f97c-2ed5-401f-a4a5-0f7d688bede7 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:26.207373 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63c508a-6089-444f-bdf0-c68dd339cb4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:26.208039 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63c508a-6089-444f-bdf0-c68dd339cb4c None admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:26.208279 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63c508a-6089-444f-bdf0-c68dd339cb4c None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:26.208551 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63c508a-6089-444f-bdf0-c68dd339cb4c None admin] PATH_INFO: `/v3/projects/e78fff7f1f6e4f68b8d233ad2e5bbd4f/users/d55edf6cd1804006aa8269b4c6f80aec/roles/f21e23654e4740158c7d4f46e726e515` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:26.223269 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63c508a-6089-444f-bdf0-c68dd339cb4c None admin] RBAC: Authorizing `identity:create_grant(project_id=e78fff7f1f6e4f68b8d233ad2e5bbd4f, user_id=d55edf6cd1804006aa8269b4c6f80aec, role_id=f21e23654e4740158c7d4f46e726e515)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:26.224698 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63c508a-6089-444f-bdf0-c68dd339cb4c None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:26.237343 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 324/631] 10.222.0.102 () {58 vars in 1444 bytes} [Thu Sep 1 17:18:26 2022] PUT /identity/v3/projects/e78fff7f1f6e4f68b8d233ad2e5bbd4f/users/d55edf6cd1804006aa8269b4c6f80aec/roles/f21e23654e4740158c7d4f46e726e515 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:18:26.239135 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226bcd00-1f21-4e4b-9d51-65786a774303 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:26.239572 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226bcd00-1f21-4e4b-9d51-65786a774303 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:26.240033 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226bcd00-1f21-4e4b-9d51-65786a774303 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:26.259036 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-226bcd00-1f21-4e4b-9d51-65786a774303 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:18:26.260326 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-226bcd00-1f21-4e4b-9d51-65786a774303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:18:26.280371 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02f1c6e-f865-4d0a-b973-4cb2e5fde8b4 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:26.321904 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d01e01a-2956-4bf7-8ad5-387fa8e83f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:26.323456 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d01e01a-2956-4bf7-8ad5-387fa8e83f9f None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:26.323915 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d01e01a-2956-4bf7-8ad5-387fa8e83f9f None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:26.324334 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d01e01a-2956-4bf7-8ad5-387fa8e83f9f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:26.327125 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:26.343282 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 325/632] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:18:26 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:18:26.344400 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adb5526-721d-4864-908b-da48db156608 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:26.344677 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adb5526-721d-4864-908b-da48db156608 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:26.344944 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adb5526-721d-4864-908b-da48db156608 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:26.346406 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 326/633] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:18:26 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:18:26.351675 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adb5526-721d-4864-908b-da48db156608 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:26.398805 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:26.433343 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d01e01a-2956-4bf7-8ad5-387fa8e83f9f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:26.435544 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d01e01a-2956-4bf7-8ad5-387fa8e83f9f None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:26.477533 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adb5526-721d-4864-908b-da48db156608 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:26.523884 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75463734-78f0-48b3-b002-62d02aef6366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:26.524494 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75463734-78f0-48b3-b002-62d02aef6366 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:26.524776 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75463734-78f0-48b3-b002-62d02aef6366 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:26.525046 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75463734-78f0-48b3-b002-62d02aef6366 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:26.540471 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75463734-78f0-48b3-b002-62d02aef6366 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:18:26.547578 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-75463734-78f0-48b3-b002-62d02aef6366 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:18:26.547578 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:26.547578 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:26.547578 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:26.547578 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:26.547578 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:26.547578 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:26.547578 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:26.547578 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:26.547578 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:26.547578 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:26.547578 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:26.547578 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:18:26.547578 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:18:26.547578 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:18:26.547578 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:18:26.548500 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:18:26.548500 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:18:26.548500 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:18:26.548500 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:18:26.548500 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:26.548500 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:26.548500 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:26.548500 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:26.548500 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:26.548500 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:26.548500 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:26.548500 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:26.548500 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:26.548500 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:26.548500 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:26.548500 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:26.549352 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:26.549352 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:26.549352 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:26.549352 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:26.549352 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:18:26.549352 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:18:26.549352 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:18:26.549352 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:18:26.549352 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:18:26.549352 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:18:26.549352 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:18:26.549352 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:18:26.550009 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 327/634] 10.222.0.102 () {56 vars in 1107 bytes} [Thu Sep 1 17:18:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:26.553328 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b91774-1fff-4728-868a-75920f3bb8f7 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:26.553328 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b91774-1fff-4728-868a-75920f3bb8f7 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:26.553328 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b91774-1fff-4728-868a-75920f3bb8f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:26.558408 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 308/635] 10.222.0.102 () {58 vars in 1287 bytes} [Thu Sep 1 17:18:26 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 17:18:26.559312 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d01e01a-2956-4bf7-8ad5-387fa8e83f9f None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:26.574008 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-a5b91774-1fff-4728-868a-75920f3bb8f7 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:18:26.575968 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-a5b91774-1fff-4728-868a-75920f3bb8f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:18:26.641379 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:26.646312 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:26.666779 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 328/636] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:18:26 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:18:26.668010 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d01e01a-2956-4bf7-8ad5-387fa8e83f9f None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:26.674324 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b91774-1fff-4728-868a-75920f3bb8f7 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:26.715937 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d001352-ece0-4271-937a-41748df06aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:26.716544 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d001352-ece0-4271-937a-41748df06aed None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:26.716824 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d001352-ece0-4271-937a-41748df06aed None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:26.717096 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d001352-ece0-4271-937a-41748df06aed None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:26.717783 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d001352-ece0-4271-937a-41748df06aed None admin] RBAC: Adding query filter params (name=service) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:26.718085 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d001352-ece0-4271-937a-41748df06aed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:26.719454 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d001352-ece0-4271-937a-41748df06aed None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:26.729675 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 309/637] 10.222.0.102 () {56 vars in 1108 bytes} [Thu Sep 1 17:18:26 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:26.730611 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d001352-ece0-4271-937a-41748df06aed None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:26.799203 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b91774-1fff-4728-868a-75920f3bb8f7 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:26.843336 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f79076-f123-4326-a1a3-9848d67f57c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:26.843972 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f79076-f123-4326-a1a3-9848d67f57c9 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:26.844403 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f79076-f123-4326-a1a3-9848d67f57c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:26.844683 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f79076-f123-4326-a1a3-9848d67f57c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:26.854774 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d001352-ece0-4271-937a-41748df06aed None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:26.861221 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f79076-f123-4326-a1a3-9848d67f57c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:26.862711 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f79076-f123-4326-a1a3-9848d67f57c9 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:26.883184 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-94f79076-f123-4326-a1a3-9848d67f57c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:26.883184 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:26.883184 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:26.883184 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:26.883184 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:26.883184 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:26.883184 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:26.883184 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:26.883184 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:26.883184 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:26.883184 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:26.883184 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:26.883184 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:18:26.883184 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:18:26.883184 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:18:26.883184 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:18:26.884177 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:26.884177 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:26.884177 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:26.884177 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:26.884177 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:26.884177 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:26.884177 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:26.884177 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:26.884177 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:26.884177 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:26.884177 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:26.884177 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:26.884177 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:26.884177 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:26.884177 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:26.884177 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:26.884177 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:18:26.885062 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:18:26.885062 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:26.885062 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:18:26.885233 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 329/638] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:18:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:26.889345 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f79076-f123-4326-a1a3-9848d67f57c9 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:26.896349 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecab0c47-b7f8-4809-b2c3-64e39af4737a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:26.897063 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecab0c47-b7f8-4809-b2c3-64e39af4737a None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:26.897435 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecab0c47-b7f8-4809-b2c3-64e39af4737a None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:26.897767 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecab0c47-b7f8-4809-b2c3-64e39af4737a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:26.905522 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:26.912255 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecab0c47-b7f8-4809-b2c3-64e39af4737a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:18:26.917282 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-ecab0c47-b7f8-4809-b2c3-64e39af4737a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:18:26.917282 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:26.917282 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:26.917282 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:26.917282 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:26.917282 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:26.917282 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:26.917282 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:26.917282 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:26.917282 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:26.917282 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:26.917282 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:26.917282 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:18:26.917282 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:18:26.917282 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:18:26.917282 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:18:26.918410 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:18:26.918410 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:18:26.918410 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:18:26.918410 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:18:26.918410 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:26.918410 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:26.918410 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:26.918410 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:26.918410 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:26.918410 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:26.918410 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:26.918410 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:26.918410 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:26.918410 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:26.918410 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:26.918410 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:26.919359 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:26.919359 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:26.919359 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:26.919359 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:26.919359 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:18:26.919359 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:18:26.919359 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:18:26.919359 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:18:26.919359 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:18:26.919359 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:18:26.919359 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:18:26.919359 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:18:26.921536 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 310/639] 10.222.0.102 () {56 vars in 1107 bytes} [Thu Sep 1 17:18:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:26.922721 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecab0c47-b7f8-4809-b2c3-64e39af4737a None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:26.924126 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:26.998283 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f79076-f123-4326-a1a3-9848d67f57c9 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:27.034795 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01110b3b-7380-464f-a36a-83b0ad3d0888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:27.035458 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01110b3b-7380-464f-a36a-83b0ad3d0888 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:27.035740 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01110b3b-7380-464f-a36a-83b0ad3d0888 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:27.036013 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01110b3b-7380-464f-a36a-83b0ad3d0888 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:27.036172 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecab0c47-b7f8-4809-b2c3-64e39af4737a None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:27.047169 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01110b3b-7380-464f-a36a-83b0ad3d0888 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:27.048631 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01110b3b-7380-464f-a36a-83b0ad3d0888 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:27.064545 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-01110b3b-7380-464f-a36a-83b0ad3d0888 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:27.064545 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:27.064545 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:27.064545 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:27.064545 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:27.064545 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:27.064545 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:27.064545 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:27.064545 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:27.064545 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:27.064545 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:27.064545 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:27.064545 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:18:27.064545 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:18:27.064545 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:18:27.064545 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:18:27.065654 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:27.065654 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:27.065654 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:27.065654 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:27.065654 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:27.065654 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:27.065654 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:27.065654 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:27.065654 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:27.065654 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:27.065654 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:27.065654 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:27.065654 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:27.065654 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:27.065654 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:27.065654 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:27.065654 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:18:27.067099 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:18:27.067099 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:27.067099 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:18:27.067099 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 330/640] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:18:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:27.067099 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01110b3b-7380-464f-a36a-83b0ad3d0888 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:27.080376 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f201d898-37b5-4c13-ac4a-4cc464f0792a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:27.080987 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f201d898-37b5-4c13-ac4a-4cc464f0792a None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:27.081271 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f201d898-37b5-4c13-ac4a-4cc464f0792a None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:27.081600 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f201d898-37b5-4c13-ac4a-4cc464f0792a None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:27.082295 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f201d898-37b5-4c13-ac4a-4cc464f0792a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:27.082599 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f201d898-37b5-4c13-ac4a-4cc464f0792a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:27.083969 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f201d898-37b5-4c13-ac4a-4cc464f0792a None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:27.096375 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 311/641] 10.222.0.102 () {56 vars in 1114 bytes} [Thu Sep 1 17:18:26 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:27.097193 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f201d898-37b5-4c13-ac4a-4cc464f0792a None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:27.161532 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:27.167444 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:27.186165 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01110b3b-7380-464f-a36a-83b0ad3d0888 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:27.208135 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f201d898-37b5-4c13-ac4a-4cc464f0792a None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:27.224375 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39353b12-a34e-489b-9544-d454209400db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:27.225099 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39353b12-a34e-489b-9544-d454209400db None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:27.225412 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39353b12-a34e-489b-9544-d454209400db None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:27.225604 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39353b12-a34e-489b-9544-d454209400db None admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:27.226381 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39353b12-a34e-489b-9544-d454209400db None admin] RBAC: Adding query filter params (name=service) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:27.226692 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39353b12-a34e-489b-9544-d454209400db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:27.228393 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39353b12-a34e-489b-9544-d454209400db None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:27.241419 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 331/642] 10.222.0.102 () {56 vars in 1108 bytes} [Thu Sep 1 17:18:27 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:27.241419 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39353b12-a34e-489b-9544-d454209400db None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:27.248740 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79321d20-5072-43ba-b4dd-645715bfaf67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:27.249415 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79321d20-5072-43ba-b4dd-645715bfaf67 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:27.249701 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79321d20-5072-43ba-b4dd-645715bfaf67 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:27.249972 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79321d20-5072-43ba-b4dd-645715bfaf67 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:27.261421 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79321d20-5072-43ba-b4dd-645715bfaf67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:18:27.266191 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-79321d20-5072-43ba-b4dd-645715bfaf67 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:18:27.266191 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:27.266191 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:27.266191 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:27.266191 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:27.266191 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:27.266191 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:27.266191 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:27.266191 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:27.266191 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:27.266191 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:27.266191 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:27.266191 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:18:27.266191 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:18:27.266191 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:18:27.266191 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:18:27.267139 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:18:27.267139 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:18:27.267139 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:18:27.267139 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:18:27.267139 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:27.267139 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:27.267139 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:27.267139 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:27.267139 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:27.267139 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:27.267139 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:27.267139 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:27.267139 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:27.267139 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:27.267139 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:27.267139 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:27.267959 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:27.267959 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:27.267959 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:27.267959 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:27.267959 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:18:27.267959 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:18:27.267959 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:18:27.267959 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:18:27.267959 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:18:27.267959 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:18:27.267959 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:18:27.267959 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:18:27.267959 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 312/643] 10.222.0.102 () {56 vars in 1107 bytes} [Thu Sep 1 17:18:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:27.268638 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79321d20-5072-43ba-b4dd-645715bfaf67 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:27.357161 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39353b12-a34e-489b-9544-d454209400db None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:27.380933 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79321d20-5072-43ba-b4dd-645715bfaf67 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:27.393787 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d118aa5a-67b6-4480-bef7-1b3d97f0c40d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:27.395032 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d118aa5a-67b6-4480-bef7-1b3d97f0c40d None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:27.395312 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d118aa5a-67b6-4480-bef7-1b3d97f0c40d None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:27.395578 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d118aa5a-67b6-4480-bef7-1b3d97f0c40d None admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:27.396225 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d118aa5a-67b6-4480-bef7-1b3d97f0c40d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:27.396509 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d118aa5a-67b6-4480-bef7-1b3d97f0c40d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:27.397908 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d118aa5a-67b6-4480-bef7-1b3d97f0c40d None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:27.401032 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:27.409987 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 332/644] 10.222.0.102 () {56 vars in 1114 bytes} [Thu Sep 1 17:18:27 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:27.414456 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d118aa5a-67b6-4480-bef7-1b3d97f0c40d None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:27.420223 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced23b72-686b-4f3e-8d56-1a8f9ff1fc98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:27.420641 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced23b72-686b-4f3e-8d56-1a8f9ff1fc98 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:27.420920 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced23b72-686b-4f3e-8d56-1a8f9ff1fc98 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:27.421189 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced23b72-686b-4f3e-8d56-1a8f9ff1fc98 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:27.428188 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced23b72-686b-4f3e-8d56-1a8f9ff1fc98 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:27.429073 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:27.429654 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced23b72-686b-4f3e-8d56-1a8f9ff1fc98 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:27.441846 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-ced23b72-686b-4f3e-8d56-1a8f9ff1fc98 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:18:27.441846 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:27.441846 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:27.441846 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:27.441846 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:27.441846 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:27.441846 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:27.441846 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:27.441846 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:27.441846 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:27.441846 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:27.441846 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:27.441846 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:18:27.441846 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:18:27.441846 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:18:27.443354 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:18:27.443354 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:27.443354 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:27.443354 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:27.443354 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:27.443354 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:27.443354 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:27.443354 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:27.443354 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:27.443354 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:27.443354 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:27.443354 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:27.443354 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:27.443354 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:27.443354 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:27.443354 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:27.443354 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:27.444434 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:18:27.444434 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:18:27.444434 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:18:27.444434 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:18:27.444434 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:18:27.444434 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:18:27.444434 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:18:27.444434 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:18:27.448539 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 313/645] 10.222.0.102 () {56 vars in 1116 bytes} [Thu Sep 1 17:18:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:27.449426 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced23b72-686b-4f3e-8d56-1a8f9ff1fc98 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:27.531154 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d118aa5a-67b6-4480-bef7-1b3d97f0c40d None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:27.557594 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced23b72-686b-4f3e-8d56-1a8f9ff1fc98 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:27.565958 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce51e75d-1d7b-4435-8da7-213bfff310b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:27.567166 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce51e75d-1d7b-4435-8da7-213bfff310b3 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:27.567540 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce51e75d-1d7b-4435-8da7-213bfff310b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:27.567905 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce51e75d-1d7b-4435-8da7-213bfff310b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:27.578146 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce51e75d-1d7b-4435-8da7-213bfff310b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:27.579708 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce51e75d-1d7b-4435-8da7-213bfff310b3 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:27.594485 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e6bfc9-56c7-48b7-abb9-d31e6d037c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:27.594961 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-ce51e75d-1d7b-4435-8da7-213bfff310b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:27.594961 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:27.594961 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:27.594961 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:27.594961 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:27.594961 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:27.594961 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:27.594961 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:27.594961 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:27.594961 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:27.594961 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:27.594961 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:27.594961 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:18:27.594961 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:18:27.594961 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:18:27.594961 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:18:27.596021 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:27.596021 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:27.596021 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:27.596021 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:27.596021 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:27.596021 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:27.596021 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:27.596021 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:27.596021 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:27.596021 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:27.596021 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:27.596021 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:27.596021 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:27.596021 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:27.596021 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:27.596021 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:27.596021 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:18:27.597116 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:18:27.597116 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:27.597116 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:18:27.597435 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e6bfc9-56c7-48b7-abb9-d31e6d037c42 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:27.597435 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e6bfc9-56c7-48b7-abb9-d31e6d037c42 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:27.597435 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e6bfc9-56c7-48b7-abb9-d31e6d037c42 None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:27.602818 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 333/646] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:18:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:27.603200 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e6bfc9-56c7-48b7-abb9-d31e6d037c42 None admin] RBAC: Adding query filter params (name=service) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:27.603200 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e6bfc9-56c7-48b7-abb9-d31e6d037c42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:27.603381 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce51e75d-1d7b-4435-8da7-213bfff310b3 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:27.603474 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e6bfc9-56c7-48b7-abb9-d31e6d037c42 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:27.613093 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 314/647] 10.222.0.102 () {56 vars in 1108 bytes} [Thu Sep 1 17:18:27 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:27.618110 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e6bfc9-56c7-48b7-abb9-d31e6d037c42 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:27.664676 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:27.665485 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:27.716705 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce51e75d-1d7b-4435-8da7-213bfff310b3 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:27.730692 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e6bfc9-56c7-48b7-abb9-d31e6d037c42 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:27.751805 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5202bb58-24c1-42a9-9be0-a6f33e44eb56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:27.752431 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5202bb58-24c1-42a9-9be0-a6f33e44eb56 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:27.752849 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5202bb58-24c1-42a9-9be0-a6f33e44eb56 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:27.753180 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5202bb58-24c1-42a9-9be0-a6f33e44eb56 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:27.762071 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5202bb58-24c1-42a9-9be0-a6f33e44eb56 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:27.763493 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5202bb58-24c1-42a9-9be0-a6f33e44eb56 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:27.766563 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b16d56-bc91-4a32-ad7b-de45e27a662b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:27.767397 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b16d56-bc91-4a32-ad7b-de45e27a662b None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:27.767892 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b16d56-bc91-4a32-ad7b-de45e27a662b None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:27.768373 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b16d56-bc91-4a32-ad7b-de45e27a662b None admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:27.770051 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b16d56-bc91-4a32-ad7b-de45e27a662b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:27.770562 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b16d56-bc91-4a32-ad7b-de45e27a662b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:27.772171 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b16d56-bc91-4a32-ad7b-de45e27a662b None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:27.774959 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-5202bb58-24c1-42a9-9be0-a6f33e44eb56 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 01 17:18:27.774959 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:27.774959 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:27.774959 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:27.774959 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:27.774959 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:27.774959 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:27.774959 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:27.774959 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:27.774959 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:27.774959 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:27.774959 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:27.774959 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:18:27.774959 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:18:27.774959 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:18:27.774959 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:18:27.775899 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:27.775899 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:27.775899 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:18:27.775899 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:18:27.775899 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:18:27.775899 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:18:27.775899 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:27.775899 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:27.775899 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:27.775899 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:27.775899 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:27.775899 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:27.775899 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:27.775899 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:27.775899 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:27.775899 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:27.775899 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:27.775899 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:27.776850 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:27.776850 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:27.776850 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:18:27.776850 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:18:27.776850 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:18:27.776850 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:18:27.776850 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:18:27.776850 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:18:27.776850 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 01 17:18:27.776850 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:18:27.776850 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 334/648] 10.222.0.102 () {56 vars in 1104 bytes} [Thu Sep 1 17:18:27 2022] GET /identity/v3/users/glance => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:27.776850 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5202bb58-24c1-42a9-9be0-a6f33e44eb56 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:27.787562 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 315/649] 10.222.0.102 () {56 vars in 1153 bytes} [Thu Sep 1 17:18:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:27.787562 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b16d56-bc91-4a32-ad7b-de45e27a662b None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:27.889418 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5202bb58-24c1-42a9-9be0-a6f33e44eb56 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:27.901767 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:27.905628 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b16d56-bc91-4a32-ad7b-de45e27a662b None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:27.910400 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:27.926338 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b43c56-3864-46bf-9b48-57f6b35a99e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:27.926943 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b43c56-3864-46bf-9b48-57f6b35a99e6 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:27.927219 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b43c56-3864-46bf-9b48-57f6b35a99e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:27.927487 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b43c56-3864-46bf-9b48-57f6b35a99e6 None admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:27.928123 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b43c56-3864-46bf-9b48-57f6b35a99e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:27.928424 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b43c56-3864-46bf-9b48-57f6b35a99e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:27.929804 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b43c56-3864-46bf-9b48-57f6b35a99e6 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:27.942690 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 335/650] 10.222.0.102 () {56 vars in 1114 bytes} [Thu Sep 1 17:18:27 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:27.947245 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b43c56-3864-46bf-9b48-57f6b35a99e6 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:27.948159 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0edf712-1413-4b89-9cf7-2c3de6236b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:27.948882 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0edf712-1413-4b89-9cf7-2c3de6236b7f None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:27.949232 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0edf712-1413-4b89-9cf7-2c3de6236b7f None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:27.949639 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0edf712-1413-4b89-9cf7-2c3de6236b7f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:27.961112 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0edf712-1413-4b89-9cf7-2c3de6236b7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:27.962612 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0edf712-1413-4b89-9cf7-2c3de6236b7f None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:27.977278 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-e0edf712-1413-4b89-9cf7-2c3de6236b7f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:27.977278 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:27.977278 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:27.977278 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:27.977278 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:27.977278 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:27.977278 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:27.977278 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:27.977278 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:27.977278 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:27.977278 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:27.977278 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:27.977278 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:18:27.977278 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:18:27.977278 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:18:27.977278 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:18:27.978671 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:27.978671 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:27.978671 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:27.978671 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:27.978671 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:27.978671 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:27.978671 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:27.978671 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:27.978671 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:27.978671 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:27.978671 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:27.978671 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:27.978671 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:27.978671 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:27.978671 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:27.978671 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:27.978671 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:18:27.979736 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:18:27.979736 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:27.979736 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:18:27.983323 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 316/651] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:18:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:27.984216 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0edf712-1413-4b89-9cf7-2c3de6236b7f None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:28.055665 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b43c56-3864-46bf-9b48-57f6b35a99e6 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:28.090147 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0edf712-1413-4b89-9cf7-2c3de6236b7f None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:28.096103 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cc4a29-f8ca-4744-95ab-d437b14ef37e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:28.097064 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cc4a29-f8ca-4744-95ab-d437b14ef37e None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:28.097373 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cc4a29-f8ca-4744-95ab-d437b14ef37e None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:28.097648 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cc4a29-f8ca-4744-95ab-d437b14ef37e None admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:28.098374 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cc4a29-f8ca-4744-95ab-d437b14ef37e None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:28.098682 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cc4a29-f8ca-4744-95ab-d437b14ef37e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:28.100252 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cc4a29-f8ca-4744-95ab-d437b14ef37e None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:28.123931 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 336/652] 10.222.0.102 () {56 vars in 1142 bytes} [Thu Sep 1 17:18:27 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 374 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:28.124869 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cc4a29-f8ca-4744-95ab-d437b14ef37e None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:28.124986 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d891de4-bc17-487e-ae77-b6c2cc7cdc48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:28.125609 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d891de4-bc17-487e-ae77-b6c2cc7cdc48 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:28.125888 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d891de4-bc17-487e-ae77-b6c2cc7cdc48 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:28.126158 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d891de4-bc17-487e-ae77-b6c2cc7cdc48 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:28.126931 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d891de4-bc17-487e-ae77-b6c2cc7cdc48 None admin] RBAC: Adding query filter params (role.id=f21e23654e4740158c7d4f46e726e515, scope.project.id=e78fff7f1f6e4f68b8d233ad2e5bbd4f, user.id=5f8293316baf4774b62fc11d55d50865) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:28.127243 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d891de4-bc17-487e-ae77-b6c2cc7cdc48 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:28.129231 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d891de4-bc17-487e-ae77-b6c2cc7cdc48 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:28.135523 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 317/653] 10.222.0.102 () {56 vars in 1379 bytes} [Thu Sep 1 17:18:27 2022] GET /identity/v3/role_assignments?user.id=5f8293316baf4774b62fc11d55d50865&scope.project.id=e78fff7f1f6e4f68b8d233ad2e5bbd4f&role.id=f21e23654e4740158c7d4f46e726e515 => generated 610 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:28.136326 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d891de4-bc17-487e-ae77-b6c2cc7cdc48 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:28.165072 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:28.188013 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:28.258237 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cc4a29-f8ca-4744-95ab-d437b14ef37e None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:28.259308 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d891de4-bc17-487e-ae77-b6c2cc7cdc48 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:28.297051 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0ea166-7cc2-4b51-9caf-0a6deb30ab8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:28.297799 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0ea166-7cc2-4b51-9caf-0a6deb30ab8e None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:28.299098 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0ea166-7cc2-4b51-9caf-0a6deb30ab8e None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:28.299098 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0ea166-7cc2-4b51-9caf-0a6deb30ab8e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:28.299285 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2443f9c4-6d8e-4ca4-b2ea-012fe4b5bfcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:28.299285 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2443f9c4-6d8e-4ca4-b2ea-012fe4b5bfcf None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:28.299446 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2443f9c4-6d8e-4ca4-b2ea-012fe4b5bfcf None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:28.300280 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2443f9c4-6d8e-4ca4-b2ea-012fe4b5bfcf None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:28.300514 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2443f9c4-6d8e-4ca4-b2ea-012fe4b5bfcf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:28.301665 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2443f9c4-6d8e-4ca4-b2ea-012fe4b5bfcf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:28.302738 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2443f9c4-6d8e-4ca4-b2ea-012fe4b5bfcf None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:28.307791 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0ea166-7cc2-4b51-9caf-0a6deb30ab8e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:28.309221 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0ea166-7cc2-4b51-9caf-0a6deb30ab8e None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:28.316244 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 318/654] 10.222.0.102 () {56 vars in 1114 bytes} [Thu Sep 1 17:18:28 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:28.317182 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2443f9c4-6d8e-4ca4-b2ea-012fe4b5bfcf None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:28.322352 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-2d0ea166-7cc2-4b51-9caf-0a6deb30ab8e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 17:18:28.322352 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:28.322352 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:28.322352 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:28.322352 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:28.322352 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:28.322352 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:28.322352 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:28.322352 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:28.322352 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:28.322352 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:28.322352 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:28.322352 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:18:28.322352 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:18:28.322352 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:18:28.322352 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:18:28.323435 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:28.323435 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:28.323435 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:18:28.323435 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:18:28.323435 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:18:28.323435 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:18:28.323435 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:28.323435 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:28.323435 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:28.323435 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:28.323435 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:28.323435 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:28.323435 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:28.323435 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:28.323435 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:28.323435 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:28.323435 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:28.323435 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:28.324408 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:28.324408 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:28.324408 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:18:28.324408 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:18:28.324408 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:18:28.324408 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:18:28.324408 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:18:28.324408 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:18:28.324408 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 17:18:28.324408 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:18:28.326263 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 337/655] 10.222.0.102 () {56 vars in 1104 bytes} [Thu Sep 1 17:18:28 2022] GET /identity/v3/users/cinder => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:28.327432 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58bafc1-44c3-43e0-9e74-11e0d34e5ee2 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:28.327729 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58bafc1-44c3-43e0-9e74-11e0d34e5ee2 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:28.327989 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58bafc1-44c3-43e0-9e74-11e0d34e5ee2 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:28.329238 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 338/656] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:18:28 2022] GET /identity => generated 271 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:18:28.334428 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58bafc1-44c3-43e0-9e74-11e0d34e5ee2 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:28.407317 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:28.433493 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2443f9c4-6d8e-4ca4-b2ea-012fe4b5bfcf None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:28.447138 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:28.453222 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58bafc1-44c3-43e0-9e74-11e0d34e5ee2 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:28.471114 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc386b0-5a07-4826-b035-12b3528a191c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:28.472033 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc386b0-5a07-4826-b035-12b3528a191c None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:28.472033 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc386b0-5a07-4826-b035-12b3528a191c None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:28.472292 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc386b0-5a07-4826-b035-12b3528a191c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:28.491262 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c226dd30-dd50-4f15-a932-716dd0f389c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:28.491612 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c226dd30-dd50-4f15-a932-716dd0f389c2 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:28.491994 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c226dd30-dd50-4f15-a932-716dd0f389c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:28.492380 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c226dd30-dd50-4f15-a932-716dd0f389c2 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:28.499796 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c226dd30-dd50-4f15-a932-716dd0f389c2 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:28.501226 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c226dd30-dd50-4f15-a932-716dd0f389c2 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:28.513949 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-c226dd30-dd50-4f15-a932-716dd0f389c2 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 01 17:18:28.513949 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:28.513949 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:28.513949 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:28.513949 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:28.513949 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:28.513949 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:28.513949 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:28.513949 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:28.513949 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:28.513949 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:28.513949 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:28.513949 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:18:28.513949 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:18:28.513949 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:18:28.513949 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:18:28.514914 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:28.514914 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:28.514914 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:18:28.514914 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:18:28.514914 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:18:28.514914 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:18:28.514914 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:28.514914 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:28.514914 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:28.514914 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:28.514914 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:28.514914 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:28.514914 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:28.514914 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:28.514914 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:28.514914 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:28.514914 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:28.514914 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:28.515890 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:28.515890 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:28.515890 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:18:28.515890 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:18:28.515890 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:18:28.515890 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:18:28.515890 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:18:28.515890 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:18:28.515890 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 01 17:18:28.515890 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:18:28.515890 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 339/657] 10.222.0.102 () {56 vars in 1107 bytes} [Thu Sep 1 17:18:28 2022] GET /identity/v3/users/neutron => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:28.515890 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c226dd30-dd50-4f15-a932-716dd0f389c2 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:28.550709 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc386b0-5a07-4826-b035-12b3528a191c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:28.552449 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc386b0-5a07-4826-b035-12b3528a191c None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:28.625021 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c226dd30-dd50-4f15-a932-716dd0f389c2 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:28.632731 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:28.659388 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 319/658] 10.222.0.102 () {58 vars in 1287 bytes} [Thu Sep 1 17:18:28 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 17:18:28.661794 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d342084e-7eb2-4c93-b65c-f9f3651dd5c9 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:28.662081 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d342084e-7eb2-4c93-b65c-f9f3651dd5c9 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:28.662354 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d342084e-7eb2-4c93-b65c-f9f3651dd5c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:28.670777 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48afd68d-4946-4515-9043-f4f8b47c3beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:28.671516 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48afd68d-4946-4515-9043-f4f8b47c3beb None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:28.671897 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48afd68d-4946-4515-9043-f4f8b47c3beb None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:28.672272 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48afd68d-4946-4515-9043-f4f8b47c3beb None admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:28.673108 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48afd68d-4946-4515-9043-f4f8b47c3beb None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:28.673563 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48afd68d-4946-4515-9043-f4f8b47c3beb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:28.675090 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48afd68d-4946-4515-9043-f4f8b47c3beb None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:28.680966 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-d342084e-7eb2-4c93-b65c-f9f3651dd5c9 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:18:28.682203 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-d342084e-7eb2-4c93-b65c-f9f3651dd5c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:18:28.699229 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 340/659] 10.222.0.102 () {56 vars in 1142 bytes} [Thu Sep 1 17:18:28 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 374 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:28.699229 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48afd68d-4946-4515-9043-f4f8b47c3beb None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:28.732037 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:28.781377 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 320/660] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:18:28 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:18:28.781377 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d342084e-7eb2-4c93-b65c-f9f3651dd5c9 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:28.832298 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48afd68d-4946-4515-9043-f4f8b47c3beb None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:28.874890 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd725597-54e1-44e4-861c-bcf488b5ecff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:28.875778 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd725597-54e1-44e4-861c-bcf488b5ecff None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:28.876098 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd725597-54e1-44e4-861c-bcf488b5ecff None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:28.876463 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd725597-54e1-44e4-861c-bcf488b5ecff None admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:28.877403 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd725597-54e1-44e4-861c-bcf488b5ecff None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:28.877806 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd725597-54e1-44e4-861c-bcf488b5ecff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:28.879888 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd725597-54e1-44e4-861c-bcf488b5ecff None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:28.903585 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 341/661] 10.222.0.102 () {56 vars in 1144 bytes} [Thu Sep 1 17:18:28 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 376 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:28.904616 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd725597-54e1-44e4-861c-bcf488b5ecff None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:28.931596 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d342084e-7eb2-4c93-b65c-f9f3651dd5c9 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:28.942134 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:28.977797 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af500d7e-1c9b-4486-a103-e322ccce917c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:28.978603 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af500d7e-1c9b-4486-a103-e322ccce917c None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:28.978885 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af500d7e-1c9b-4486-a103-e322ccce917c None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:28.979159 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af500d7e-1c9b-4486-a103-e322ccce917c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:28.991699 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:28.993572 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af500d7e-1c9b-4486-a103-e322ccce917c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:28.995443 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af500d7e-1c9b-4486-a103-e322ccce917c None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:29.015354 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-af500d7e-1c9b-4486-a103-e322ccce917c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:29.015354 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:29.015354 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:29.015354 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:29.015354 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:29.015354 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:29.015354 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:29.015354 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:29.015354 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:29.015354 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:29.015354 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:29.015354 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:29.015354 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:18:29.015354 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:18:29.015354 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:18:29.015354 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:18:29.016349 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:29.016349 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:29.016349 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:29.016349 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:29.016349 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:29.016349 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:29.016349 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:29.016349 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:29.016349 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:29.016349 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:29.016349 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:29.016349 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:29.016349 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:29.016349 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:29.016349 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:29.016349 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:29.016349 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:18:29.017203 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:18:29.017203 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:29.017203 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:18:29.017203 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 321/662] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:18:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:29.017924 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af500d7e-1c9b-4486-a103-e322ccce917c None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:29.032582 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd725597-54e1-44e4-861c-bcf488b5ecff None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:29.077238 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0806cb-c7a5-4f2a-b6b2-949219ebe2e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:29.077874 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0806cb-c7a5-4f2a-b6b2-949219ebe2e9 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:29.078156 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0806cb-c7a5-4f2a-b6b2-949219ebe2e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:29.078428 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0806cb-c7a5-4f2a-b6b2-949219ebe2e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:29.125880 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af500d7e-1c9b-4486-a103-e322ccce917c None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:29.148927 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0806cb-c7a5-4f2a-b6b2-949219ebe2e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:29.150477 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0806cb-c7a5-4f2a-b6b2-949219ebe2e9 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:29.162344 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b47d260-fc65-4b21-ab01-6074c76e4369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:29.162965 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b47d260-fc65-4b21-ab01-6074c76e4369 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:29.163250 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b47d260-fc65-4b21-ab01-6074c76e4369 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:29.163526 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b47d260-fc65-4b21-ab01-6074c76e4369 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:29.192520 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:29.235801 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b47d260-fc65-4b21-ab01-6074c76e4369 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:29.236951 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:29.237171 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b47d260-fc65-4b21-ab01-6074c76e4369 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:29.265344 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 342/663] 10.222.0.102 () {58 vars in 1287 bytes} [Thu Sep 1 17:18:28 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 17:18:29.266195 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0806cb-c7a5-4f2a-b6b2-949219ebe2e9 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:29.350156 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 322/664] 10.222.0.102 () {58 vars in 1287 bytes} [Thu Sep 1 17:18:29 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 17:18:29.351319 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b47d260-fc65-4b21-ab01-6074c76e4369 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:29.384442 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0806cb-c7a5-4f2a-b6b2-949219ebe2e9 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:29.416568 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:29.423541 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e69e9b-eb33-4647-a3a2-fb81cb98ee69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:29.424592 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e69e9b-eb33-4647-a3a2-fb81cb98ee69 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:29.425073 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e69e9b-eb33-4647-a3a2-fb81cb98ee69 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:29.426650 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e69e9b-eb33-4647-a3a2-fb81cb98ee69 None admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:29.427855 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e69e9b-eb33-4647-a3a2-fb81cb98ee69 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:29.428319 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e69e9b-eb33-4647-a3a2-fb81cb98ee69 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:29.430966 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e69e9b-eb33-4647-a3a2-fb81cb98ee69 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:29.450177 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 343/665] 10.222.0.102 () {56 vars in 1114 bytes} [Thu Sep 1 17:18:29 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:29.450177 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e69e9b-eb33-4647-a3a2-fb81cb98ee69 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:29.464123 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b47d260-fc65-4b21-ab01-6074c76e4369 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:29.487156 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:29.508028 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddd9a02-795a-4c57-ab16-20a2b2c005c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:29.508753 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddd9a02-795a-4c57-ab16-20a2b2c005c9 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:29.509096 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddd9a02-795a-4c57-ab16-20a2b2c005c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:29.509551 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddd9a02-795a-4c57-ab16-20a2b2c005c9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:29.522828 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddd9a02-795a-4c57-ab16-20a2b2c005c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:18:29.527757 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-0ddd9a02-795a-4c57-ab16-20a2b2c005c9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:18:29.527757 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:29.527757 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:29.527757 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:29.527757 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:29.527757 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:29.527757 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:29.527757 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:29.527757 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:29.527757 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:29.527757 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:29.527757 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:29.527757 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:18:29.527757 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:18:29.527757 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:18:29.527757 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:18:29.528956 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:18:29.528956 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:18:29.528956 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:18:29.528956 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:18:29.528956 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:29.528956 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:29.528956 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:29.528956 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:29.528956 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:29.528956 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:29.528956 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:29.528956 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:29.528956 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:29.528956 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:29.528956 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:29.528956 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:29.530360 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:29.530360 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:29.530360 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:29.530360 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:29.530360 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:18:29.530360 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:18:29.530360 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:18:29.530360 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:18:29.530360 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:18:29.530360 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:18:29.530360 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:18:29.530360 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:18:29.530360 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 323/666] 10.222.0.102 () {56 vars in 1107 bytes} [Thu Sep 1 17:18:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:29.531300 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddd9a02-795a-4c57-ab16-20a2b2c005c9 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:29.573050 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e69e9b-eb33-4647-a3a2-fb81cb98ee69 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:29.609872 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0363386f-a207-4da5-aca5-ea99bbba227a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:29.610504 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0363386f-a207-4da5-aca5-ea99bbba227a None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:29.610777 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0363386f-a207-4da5-aca5-ea99bbba227a None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:29.611047 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0363386f-a207-4da5-aca5-ea99bbba227a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:29.625336 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0363386f-a207-4da5-aca5-ea99bbba227a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:29.627038 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0363386f-a207-4da5-aca5-ea99bbba227a None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:29.644198 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddd9a02-795a-4c57-ab16-20a2b2c005c9 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:29.645509 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-0363386f-a207-4da5-aca5-ea99bbba227a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:29.645509 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:29.645509 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:29.645509 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:29.645509 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:29.645509 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:29.645509 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:29.645509 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:29.645509 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:29.645509 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:29.645509 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:29.645509 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:29.645509 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:18:29.645509 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:18:29.645509 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:18:29.645509 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:18:29.646601 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:29.646601 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:29.646601 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:29.646601 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:29.646601 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:29.646601 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:29.646601 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:29.646601 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:29.646601 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:29.646601 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:29.646601 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:29.646601 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:29.646601 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:29.646601 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:29.646601 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:29.646601 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:29.646601 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:18:29.647666 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:18:29.647666 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:29.647666 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:18:29.651620 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 344/667] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:18:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:29.652516 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0363386f-a207-4da5-aca5-ea99bbba227a None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:29.666664 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:29.686920 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250b2f5d-79e2-4808-abfd-e45af2361184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:29.687712 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250b2f5d-79e2-4808-abfd-e45af2361184 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:29.687996 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250b2f5d-79e2-4808-abfd-e45af2361184 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:29.688266 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250b2f5d-79e2-4808-abfd-e45af2361184 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:29.702117 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250b2f5d-79e2-4808-abfd-e45af2361184 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:29.703811 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250b2f5d-79e2-4808-abfd-e45af2361184 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:29.723186 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-250b2f5d-79e2-4808-abfd-e45af2361184 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:29.723186 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:29.723186 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:29.723186 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:29.723186 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:29.723186 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:29.723186 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:29.723186 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:29.723186 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:29.723186 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:29.723186 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:29.723186 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:29.723186 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:18:29.723186 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:18:29.723186 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:18:29.723186 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:18:29.729368 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:29.729368 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:29.729368 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:29.729368 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:29.729368 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:29.729368 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:29.729368 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:29.729368 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:29.729368 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:29.729368 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:29.729368 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:29.729368 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:29.729368 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:29.729368 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:29.729368 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:29.729368 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:29.729368 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:18:29.732237 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:18:29.732237 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:29.732237 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:18:29.732237 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 324/668] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:18:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:29.732237 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250b2f5d-79e2-4808-abfd-e45af2361184 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:29.756155 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:29.775385 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0363386f-a207-4da5-aca5-ea99bbba227a None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:29.820590 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f64de4-4eaa-4c7b-a84d-b8142261d5bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:29.822807 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f64de4-4eaa-4c7b-a84d-b8142261d5bf None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:29.823092 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f64de4-4eaa-4c7b-a84d-b8142261d5bf None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:29.823422 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f64de4-4eaa-4c7b-a84d-b8142261d5bf None admin] PATH_INFO: `/v3/projects/service` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:29.830902 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f64de4-4eaa-4c7b-a84d-b8142261d5bf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:29.832421 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f64de4-4eaa-4c7b-a84d-b8142261d5bf None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:29.841042 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250b2f5d-79e2-4808-abfd-e45af2361184 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:29.846448 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-e2f64de4-4eaa-4c7b-a84d-b8142261d5bf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:18:29.846448 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:29.846448 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:29.846448 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:29.846448 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:29.846448 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:29.846448 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:29.846448 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:29.846448 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:29.846448 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:29.846448 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:29.846448 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:29.846448 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:18:29.846448 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:18:29.846448 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:18:29.848061 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:18:29.848061 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:29.848061 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:29.848061 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:29.848061 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:29.848061 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:29.848061 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:29.848061 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:29.848061 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:29.848061 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:29.848061 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:29.848061 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:29.848061 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:29.848061 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:29.848061 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:29.848061 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:29.848061 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:29.849144 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:18:29.849144 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:18:29.849144 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:18:29.849144 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:18:29.849144 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:18:29.849144 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:18:29.849144 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:18:29.849144 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:18:29.853211 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 345/669] 10.222.0.102 () {56 vars in 1116 bytes} [Thu Sep 1 17:18:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:29.854215 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f64de4-4eaa-4c7b-a84d-b8142261d5bf None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:29.882834 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97319aa3-db10-46a2-8a27-c163809e6973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:29.883714 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97319aa3-db10-46a2-8a27-c163809e6973 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:29.884161 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97319aa3-db10-46a2-8a27-c163809e6973 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:29.884661 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97319aa3-db10-46a2-8a27-c163809e6973 None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:29.885608 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97319aa3-db10-46a2-8a27-c163809e6973 None admin] RBAC: Adding query filter params (name=service) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:29.886069 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97319aa3-db10-46a2-8a27-c163809e6973 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:29.887719 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97319aa3-db10-46a2-8a27-c163809e6973 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:29.897815 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 325/670] 10.222.0.102 () {56 vars in 1108 bytes} [Thu Sep 1 17:18:29 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:29.898867 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97319aa3-db10-46a2-8a27-c163809e6973 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:29.934992 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:29.965149 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f64de4-4eaa-4c7b-a84d-b8142261d5bf None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:30.003898 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fd8aee-0371-4432-a06a-31cfe92fa203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:30.004511 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fd8aee-0371-4432-a06a-31cfe92fa203 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:30.004793 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fd8aee-0371-4432-a06a-31cfe92fa203 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:30.005072 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fd8aee-0371-4432-a06a-31cfe92fa203 None admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:30.005749 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fd8aee-0371-4432-a06a-31cfe92fa203 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:30.006057 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fd8aee-0371-4432-a06a-31cfe92fa203 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:30.007499 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fd8aee-0371-4432-a06a-31cfe92fa203 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:30.013974 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97319aa3-db10-46a2-8a27-c163809e6973 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:30.024501 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 346/671] 10.222.0.102 () {56 vars in 1114 bytes} [Thu Sep 1 17:18:29 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:30.024501 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fd8aee-0371-4432-a06a-31cfe92fa203 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:30.033193 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:30.053806 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30055b4b-69bc-4f35-95e3-3cc7b5e454e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:30.054527 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30055b4b-69bc-4f35-95e3-3cc7b5e454e1 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:30.054806 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30055b4b-69bc-4f35-95e3-3cc7b5e454e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:30.055175 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30055b4b-69bc-4f35-95e3-3cc7b5e454e1 None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:30.055696 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30055b4b-69bc-4f35-95e3-3cc7b5e454e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:30.055998 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30055b4b-69bc-4f35-95e3-3cc7b5e454e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:30.057403 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30055b4b-69bc-4f35-95e3-3cc7b5e454e1 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:30.071597 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 326/672] 10.222.0.102 () {56 vars in 1114 bytes} [Thu Sep 1 17:18:29 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:30.072444 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30055b4b-69bc-4f35-95e3-3cc7b5e454e1 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:30.143973 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fd8aee-0371-4432-a06a-31cfe92fa203 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:30.181447 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:30.185404 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e0a69e-7831-4808-aac0-f62c42963866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:30.186092 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e0a69e-7831-4808-aac0-f62c42963866 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:30.186432 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e0a69e-7831-4808-aac0-f62c42963866 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:30.186786 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e0a69e-7831-4808-aac0-f62c42963866 None admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:30.187574 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e0a69e-7831-4808-aac0-f62c42963866 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:30.187884 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e0a69e-7831-4808-aac0-f62c42963866 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:30.189336 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e0a69e-7831-4808-aac0-f62c42963866 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:30.192195 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30055b4b-69bc-4f35-95e3-3cc7b5e454e1 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:30.202252 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 347/673] 10.222.0.102 () {56 vars in 1153 bytes} [Thu Sep 1 17:18:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:30.203093 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e0a69e-7831-4808-aac0-f62c42963866 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:30.237587 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253a0d7b-218b-445a-899f-2afb908f3457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:30.238252 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253a0d7b-218b-445a-899f-2afb908f3457 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:30.238534 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253a0d7b-218b-445a-899f-2afb908f3457 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:30.238810 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253a0d7b-218b-445a-899f-2afb908f3457 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:30.249749 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253a0d7b-218b-445a-899f-2afb908f3457 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:30.251210 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253a0d7b-218b-445a-899f-2afb908f3457 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:30.267306 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-253a0d7b-218b-445a-899f-2afb908f3457 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:30.267306 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:30.267306 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:30.267306 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:30.267306 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:30.267306 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:30.267306 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:30.267306 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:30.267306 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:30.267306 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:30.267306 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:30.267306 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:30.267306 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:18:30.267306 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:18:30.267306 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:18:30.267306 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:18:30.268427 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:30.268427 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:30.268427 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:30.268427 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:30.268427 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:30.268427 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:30.268427 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:30.268427 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:30.268427 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:30.268427 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:30.268427 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:30.268427 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:30.268427 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:30.268427 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:30.268427 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:30.268427 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:30.268427 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:18:30.270341 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:18:30.270341 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:30.270341 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:18:30.270341 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 327/674] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:18:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:30.270341 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253a0d7b-218b-445a-899f-2afb908f3457 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:30.272666 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:30.308695 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e0a69e-7831-4808-aac0-f62c42963866 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:30.345375 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093ff525-8967-422c-b8a1-dd3fda6a5cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:30.346002 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093ff525-8967-422c-b8a1-dd3fda6a5cf3 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:30.346283 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093ff525-8967-422c-b8a1-dd3fda6a5cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:30.346557 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093ff525-8967-422c-b8a1-dd3fda6a5cf3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:30.354190 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093ff525-8967-422c-b8a1-dd3fda6a5cf3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:30.355623 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093ff525-8967-422c-b8a1-dd3fda6a5cf3 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:30.366979 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-093ff525-8967-422c-b8a1-dd3fda6a5cf3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:18:30.366979 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:30.366979 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:30.366979 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:30.366979 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:30.366979 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:30.366979 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:30.366979 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:30.366979 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:30.366979 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:30.366979 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:30.366979 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:30.366979 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:18:30.366979 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:18:30.366979 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:18:30.368036 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:18:30.368036 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:30.368036 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:30.368036 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:30.368036 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:30.368036 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:30.368036 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:30.368036 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:30.368036 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:30.368036 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:30.368036 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:30.368036 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:30.368036 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:30.368036 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:30.368036 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:30.368036 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:30.368036 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:30.369214 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:18:30.369214 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:18:30.369214 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:18:30.369214 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:18:30.369214 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:18:30.369214 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:18:30.369214 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:18:30.369214 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:18:30.372744 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 348/675] 10.222.0.102 () {56 vars in 1116 bytes} [Thu Sep 1 17:18:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:30.373941 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0bc788-e33e-4d9a-9cf4-51369814b224 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:30.374229 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0bc788-e33e-4d9a-9cf4-51369814b224 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:30.374500 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0bc788-e33e-4d9a-9cf4-51369814b224 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:30.375738 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 349/676] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:18:30 2022] GET /identity => generated 271 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:18:30.376424 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0bc788-e33e-4d9a-9cf4-51369814b224 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:30.376541 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253a0d7b-218b-445a-899f-2afb908f3457 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:30.411683 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fda51d1-773c-4ab2-ab44-574a8a9fc13c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:30.412314 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fda51d1-773c-4ab2-ab44-574a8a9fc13c None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:30.412594 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fda51d1-773c-4ab2-ab44-574a8a9fc13c None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:30.412867 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fda51d1-773c-4ab2-ab44-574a8a9fc13c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:30.420021 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fda51d1-773c-4ab2-ab44-574a8a9fc13c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:30.421512 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fda51d1-773c-4ab2-ab44-574a8a9fc13c None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:30.433473 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-4fda51d1-773c-4ab2-ab44-574a8a9fc13c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:18:30.433473 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:30.433473 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:30.433473 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:30.433473 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:30.433473 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:30.433473 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:30.433473 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:30.433473 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:30.433473 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:30.433473 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:30.433473 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:30.433473 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:18:30.433473 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:18:30.433473 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:18:30.434557 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:18:30.434557 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:30.434557 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:30.434557 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:30.434557 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:30.434557 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:30.434557 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:30.434557 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:30.434557 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:30.434557 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:30.434557 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:30.434557 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:30.434557 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:30.434557 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:30.434557 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:30.434557 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:30.434557 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:30.435641 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:18:30.435641 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:18:30.435641 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:18:30.435641 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:18:30.435641 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:18:30.435641 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:18:30.435641 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:18:30.435641 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:18:30.435641 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 328/677] 10.222.0.102 () {56 vars in 1116 bytes} [Thu Sep 1 17:18:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:30.436157 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fda51d1-773c-4ab2-ab44-574a8a9fc13c None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:30.439674 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:30.482661 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0bc788-e33e-4d9a-9cf4-51369814b224 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:30.509328 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:30.519981 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0237873-0e73-485f-984a-be09c428b5ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:30.520520 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0237873-0e73-485f-984a-be09c428b5ed None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:30.520797 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0237873-0e73-485f-984a-be09c428b5ed None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:30.521074 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0237873-0e73-485f-984a-be09c428b5ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:30.521829 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0237873-0e73-485f-984a-be09c428b5ed None admin] RBAC: Adding query filter params (role.id=f21e23654e4740158c7d4f46e726e515, scope.project.id=e78fff7f1f6e4f68b8d233ad2e5bbd4f, user.id=039683b8d1d146c6add5ffa235d06f91) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:30.522141 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0237873-0e73-485f-984a-be09c428b5ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:30.523560 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0237873-0e73-485f-984a-be09c428b5ed None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:30.529185 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 350/678] 10.222.0.102 () {56 vars in 1379 bytes} [Thu Sep 1 17:18:30 2022] GET /identity/v3/role_assignments?user.id=039683b8d1d146c6add5ffa235d06f91&scope.project.id=e78fff7f1f6e4f68b8d233ad2e5bbd4f&role.id=f21e23654e4740158c7d4f46e726e515 => generated 610 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:30.530014 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0237873-0e73-485f-984a-be09c428b5ed None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:30.551544 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fda51d1-773c-4ab2-ab44-574a8a9fc13c None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:30.592792 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c155125-175e-4397-b16d-560557cb3ccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:30.593540 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c155125-175e-4397-b16d-560557cb3ccd None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:30.593821 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c155125-175e-4397-b16d-560557cb3ccd None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:30.594101 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c155125-175e-4397-b16d-560557cb3ccd None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:30.594769 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c155125-175e-4397-b16d-560557cb3ccd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:30.595079 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c155125-175e-4397-b16d-560557cb3ccd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:30.596517 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c155125-175e-4397-b16d-560557cb3ccd None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:30.610151 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 329/679] 10.222.0.102 () {56 vars in 1114 bytes} [Thu Sep 1 17:18:30 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:30.611476 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59097302-d6bb-4e77-afc5-1216672def8b None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:30.611766 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59097302-d6bb-4e77-afc5-1216672def8b None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:30.612045 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59097302-d6bb-4e77-afc5-1216672def8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:30.631232 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-59097302-d6bb-4e77-afc5-1216672def8b None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:18:30.632466 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-59097302-d6bb-4e77-afc5-1216672def8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:18:30.648543 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0237873-0e73-485f-984a-be09c428b5ed None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:30.694981 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3938d97-18a6-4e81-b74c-450de610d372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:30.695595 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3938d97-18a6-4e81-b74c-450de610d372 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:30.695879 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3938d97-18a6-4e81-b74c-450de610d372 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:30.696157 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3938d97-18a6-4e81-b74c-450de610d372 None admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:30.696845 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3938d97-18a6-4e81-b74c-450de610d372 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:30.697155 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3938d97-18a6-4e81-b74c-450de610d372 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:30.698649 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3938d97-18a6-4e81-b74c-450de610d372 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:30.699575 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:30.711510 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 330/680] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:18:30 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:18:30.712756 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59097302-d6bb-4e77-afc5-1216672def8b None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:30.717788 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 351/681] 10.222.0.102 () {56 vars in 1153 bytes} [Thu Sep 1 17:18:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:30.718894 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3938d97-18a6-4e81-b74c-450de610d372 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:30.786283 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:30.840678 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59097302-d6bb-4e77-afc5-1216672def8b None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:30.844487 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3938d97-18a6-4e81-b74c-450de610d372 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:30.885679 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124f1e56-6647-4534-8eb0-865ab554c5d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:30.886791 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124f1e56-6647-4534-8eb0-865ab554c5d3 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:30.887295 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124f1e56-6647-4534-8eb0-865ab554c5d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:30.887773 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124f1e56-6647-4534-8eb0-865ab554c5d3 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:30.889603 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3488eb-cd29-456e-97b9-3fac4e449225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:30.890469 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3488eb-cd29-456e-97b9-3fac4e449225 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:30.890758 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3488eb-cd29-456e-97b9-3fac4e449225 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:30.891027 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3488eb-cd29-456e-97b9-3fac4e449225 None admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:30.891716 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3488eb-cd29-456e-97b9-3fac4e449225 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:30.892055 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3488eb-cd29-456e-97b9-3fac4e449225 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:30.894129 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3488eb-cd29-456e-97b9-3fac4e449225 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:30.897851 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124f1e56-6647-4534-8eb0-865ab554c5d3 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:30.899623 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124f1e56-6647-4534-8eb0-865ab554c5d3 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:30.909150 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 331/682] 10.222.0.102 () {56 vars in 1153 bytes} [Thu Sep 1 17:18:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:30.910750 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3488eb-cd29-456e-97b9-3fac4e449225 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:30.913509 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-124f1e56-6647-4534-8eb0-865ab554c5d3 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 01 17:18:30.913509 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:30.913509 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:30.913509 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:30.913509 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:30.913509 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:30.913509 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:30.913509 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:30.913509 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:30.913509 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:30.913509 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:30.913509 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:30.913509 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:18:30.913509 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:18:30.913509 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:18:30.913509 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:18:30.914436 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:30.914436 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:30.914436 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:18:30.914436 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:18:30.914436 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:18:30.914436 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:18:30.914436 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:30.914436 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:30.914436 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:30.914436 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:30.914436 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:30.914436 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:30.914436 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:30.914436 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:30.914436 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:30.914436 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:30.914436 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:30.914436 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:30.915366 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:30.915366 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:30.915366 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:18:30.915366 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:18:30.915366 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:18:30.915366 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:18:30.915366 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:18:30.915366 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:18:30.915366 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 01 17:18:30.915366 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:18:30.917892 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 352/683] 10.222.0.102 () {56 vars in 1101 bytes} [Thu Sep 1 17:18:30 2022] GET /identity/v3/users/swift => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:30.923243 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124f1e56-6647-4534-8eb0-865ab554c5d3 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:30.979754 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:31.028256 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3488eb-cd29-456e-97b9-3fac4e449225 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:31.050500 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124f1e56-6647-4534-8eb0-865ab554c5d3 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:31.061447 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:31.066165 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5602a4b8-452e-4038-9ff9-5a0ff226a251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:31.066865 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5602a4b8-452e-4038-9ff9-5a0ff226a251 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:31.067148 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5602a4b8-452e-4038-9ff9-5a0ff226a251 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:31.067447 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5602a4b8-452e-4038-9ff9-5a0ff226a251 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:31.070157 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5602a4b8-452e-4038-9ff9-5a0ff226a251 None admin] RBAC: Adding query filter params (role.id=f21e23654e4740158c7d4f46e726e515, scope.project.id=e78fff7f1f6e4f68b8d233ad2e5bbd4f, user.id=523ed589872a45a09b6a7675a7fb0b75) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:31.070157 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5602a4b8-452e-4038-9ff9-5a0ff226a251 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:31.070157 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5602a4b8-452e-4038-9ff9-5a0ff226a251 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:31.076043 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 332/684] 10.222.0.102 () {56 vars in 1379 bytes} [Thu Sep 1 17:18:30 2022] GET /identity/v3/role_assignments?user.id=523ed589872a45a09b6a7675a7fb0b75&scope.project.id=e78fff7f1f6e4f68b8d233ad2e5bbd4f&role.id=f21e23654e4740158c7d4f46e726e515 => generated 610 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:31.077054 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5602a4b8-452e-4038-9ff9-5a0ff226a251 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:31.104096 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c805447e-b094-45a6-a2a5-9fc7633ce2be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:31.105117 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c805447e-b094-45a6-a2a5-9fc7633ce2be None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:31.105741 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c805447e-b094-45a6-a2a5-9fc7633ce2be None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:31.106026 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c805447e-b094-45a6-a2a5-9fc7633ce2be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:31.106811 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c805447e-b094-45a6-a2a5-9fc7633ce2be None admin] RBAC: Adding query filter params (role.id=f21e23654e4740158c7d4f46e726e515, scope.project.id=e78fff7f1f6e4f68b8d233ad2e5bbd4f, user.id=7c7d3594fdab4191be1d841af43a5c79) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:31.107382 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c805447e-b094-45a6-a2a5-9fc7633ce2be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:31.109833 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c805447e-b094-45a6-a2a5-9fc7633ce2be None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:31.121501 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 353/685] 10.222.0.102 () {56 vars in 1379 bytes} [Thu Sep 1 17:18:30 2022] GET /identity/v3/role_assignments?user.id=7c7d3594fdab4191be1d841af43a5c79&scope.project.id=e78fff7f1f6e4f68b8d233ad2e5bbd4f&role.id=f21e23654e4740158c7d4f46e726e515 => generated 610 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:31.122484 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c805447e-b094-45a6-a2a5-9fc7633ce2be None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:31.275162 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5602a4b8-452e-4038-9ff9-5a0ff226a251 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:31.299123 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:31.330761 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8978d897-c7fe-4fda-9a80-f8e3e63f7ad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:31.331303 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8978d897-c7fe-4fda-9a80-f8e3e63f7ad1 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:31.332467 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8978d897-c7fe-4fda-9a80-f8e3e63f7ad1 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:31.332752 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8978d897-c7fe-4fda-9a80-f8e3e63f7ad1 None admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:31.334079 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8978d897-c7fe-4fda-9a80-f8e3e63f7ad1 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:31.336014 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c805447e-b094-45a6-a2a5-9fc7633ce2be None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:31.338299 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8978d897-c7fe-4fda-9a80-f8e3e63f7ad1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:31.340408 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8978d897-c7fe-4fda-9a80-f8e3e63f7ad1 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:31.368001 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 333/686] 10.222.0.102 () {56 vars in 1140 bytes} [Thu Sep 1 17:18:31 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 372 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:31.373835 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8978d897-c7fe-4fda-9a80-f8e3e63f7ad1 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:31.391575 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33119511-89a0-458c-97b1-0909e7c3c8ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:31.392718 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33119511-89a0-458c-97b1-0909e7c3c8ea None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:31.393009 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33119511-89a0-458c-97b1-0909e7c3c8ea None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:31.393285 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33119511-89a0-458c-97b1-0909e7c3c8ea None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:31.415746 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33119511-89a0-458c-97b1-0909e7c3c8ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:18:31.420960 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-33119511-89a0-458c-97b1-0909e7c3c8ea None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:18:31.420960 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:31.420960 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:31.420960 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:31.420960 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:31.420960 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:31.420960 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:31.420960 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:31.420960 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:31.420960 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:31.420960 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:31.420960 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:31.420960 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:18:31.420960 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:18:31.420960 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:18:31.420960 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:18:31.423233 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:18:31.423233 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:18:31.423233 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:18:31.423233 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:18:31.423233 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:31.423233 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:31.423233 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:31.423233 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:31.423233 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:31.423233 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:31.423233 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:31.423233 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:31.423233 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:31.423233 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:31.423233 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:31.423233 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:31.424239 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:31.424239 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:31.424239 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:31.424239 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:31.424239 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:18:31.424239 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:18:31.424239 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:18:31.424239 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:18:31.424239 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:18:31.424239 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:18:31.424239 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:18:31.424239 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:18:31.424239 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 354/687] 10.222.0.102 () {56 vars in 1101 bytes} [Thu Sep 1 17:18:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:31.434343 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33119511-89a0-458c-97b1-0909e7c3c8ea None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:31.498242 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:31.546525 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8978d897-c7fe-4fda-9a80-f8e3e63f7ad1 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:31.611169 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33119511-89a0-458c-97b1-0909e7c3c8ea None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:31.630042 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383177b3-2d6e-4ce6-b856-04d4748fff1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:31.631373 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383177b3-2d6e-4ce6-b856-04d4748fff1c None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:31.631873 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383177b3-2d6e-4ce6-b856-04d4748fff1c None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:31.632358 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383177b3-2d6e-4ce6-b856-04d4748fff1c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:31.661007 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb90c05f-576f-4fcf-926a-f7e29874b9e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:31.661714 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb90c05f-576f-4fcf-926a-f7e29874b9e5 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:31.661913 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb90c05f-576f-4fcf-926a-f7e29874b9e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:31.662190 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb90c05f-576f-4fcf-926a-f7e29874b9e5 None admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:31.662859 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb90c05f-576f-4fcf-926a-f7e29874b9e5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:31.663162 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb90c05f-576f-4fcf-926a-f7e29874b9e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:31.664671 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb90c05f-576f-4fcf-926a-f7e29874b9e5 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:31.685350 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 355/688] 10.222.0.102 () {56 vars in 1104 bytes} [Thu Sep 1 17:18:31 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:31.689483 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb90c05f-576f-4fcf-926a-f7e29874b9e5 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:31.702583 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:31.754981 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383177b3-2d6e-4ce6-b856-04d4748fff1c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:31.756560 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383177b3-2d6e-4ce6-b856-04d4748fff1c None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:31.818347 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:31.831140 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb90c05f-576f-4fcf-926a-f7e29874b9e5 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:31.902047 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f02d4bb-168b-4c41-bd39-5c5c9a451db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:31.902602 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f02d4bb-168b-4c41-bd39-5c5c9a451db9 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:31.902881 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f02d4bb-168b-4c41-bd39-5c5c9a451db9 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:31.903158 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f02d4bb-168b-4c41-bd39-5c5c9a451db9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:31.925355 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f02d4bb-168b-4c41-bd39-5c5c9a451db9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:31.927327 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f02d4bb-168b-4c41-bd39-5c5c9a451db9 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:31.954566 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-4f02d4bb-168b-4c41-bd39-5c5c9a451db9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:31.954566 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:31.954566 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:31.954566 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:31.954566 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:31.954566 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:31.954566 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:31.954566 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:31.954566 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:31.954566 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:31.954566 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:31.954566 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:31.954566 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:18:31.954566 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:18:31.954566 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:18:31.954566 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:18:31.955527 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:31.955527 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:31.955527 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:31.955527 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:31.955527 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:31.955527 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:31.955527 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:31.955527 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:31.955527 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:31.955527 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:31.955527 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:31.955527 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:31.955527 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:31.955527 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:31.955527 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:31.955527 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:31.955527 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:18:31.956399 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:18:31.956399 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:31.956399 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:18:31.960448 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 356/689] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:18:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:18:31.961987 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 334/690] 10.222.0.102 () {58 vars in 1287 bytes} [Thu Sep 1 17:18:31 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 589 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 17:18:31.962993 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f02d4bb-168b-4c41-bd39-5c5c9a451db9 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:31.967515 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383177b3-2d6e-4ce6-b856-04d4748fff1c None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:32.093871 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:32.120108 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f02d4bb-168b-4c41-bd39-5c5c9a451db9 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:32.121517 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383177b3-2d6e-4ce6-b856-04d4748fff1c None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:32.157226 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91aed494-e81d-4f61-8424-0497a79fd5ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:32.157814 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aed494-e81d-4f61-8424-0497a79fd5ac None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:32.158090 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aed494-e81d-4f61-8424-0497a79fd5ac None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:32.158359 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aed494-e81d-4f61-8424-0497a79fd5ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:32.169589 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91aed494-e81d-4f61-8424-0497a79fd5ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:32.171035 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91aed494-e81d-4f61-8424-0497a79fd5ac None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:32.186657 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-91aed494-e81d-4f61-8424-0497a79fd5ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:32.186657 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:32.186657 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:32.186657 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:32.186657 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:32.186657 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:32.186657 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:32.186657 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:32.186657 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:32.186657 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:32.186657 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:32.186657 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:32.186657 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:18:32.186657 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:18:32.186657 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:18:32.186657 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:18:32.187624 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:32.187624 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:32.187624 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:32.187624 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:32.187624 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:32.187624 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:32.187624 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:32.187624 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:32.187624 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:32.187624 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:32.187624 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:32.187624 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:32.187624 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:32.187624 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:32.187624 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:32.187624 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:32.187624 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:18:32.188585 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:18:32.188585 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:32.188585 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:18:32.188585 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 335/691] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:18:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:18:32.195962 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e69efa-f9e9-4457-80ca-2769b6abff3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:32.196884 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e69efa-f9e9-4457-80ca-2769b6abff3b None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:32.197408 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e69efa-f9e9-4457-80ca-2769b6abff3b None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:32.197879 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e69efa-f9e9-4457-80ca-2769b6abff3b None admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:32.198481 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91aed494-e81d-4f61-8424-0497a79fd5ac None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:32.199069 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e69efa-f9e9-4457-80ca-2769b6abff3b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:32.199576 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e69efa-f9e9-4457-80ca-2769b6abff3b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:32.201146 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e69efa-f9e9-4457-80ca-2769b6abff3b None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:32.213962 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 357/692] 10.222.0.102 () {56 vars in 1114 bytes} [Thu Sep 1 17:18:31 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:32.219594 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e69efa-f9e9-4457-80ca-2769b6abff3b None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:32.239082 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:32.374729 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91aed494-e81d-4f61-8424-0497a79fd5ac None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:32.409559 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e69efa-f9e9-4457-80ca-2769b6abff3b None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:32.428983 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:32.440146 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da50b1d9-64ee-4daa-a722-653c33986bbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:32.440764 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da50b1d9-64ee-4daa-a722-653c33986bbb None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:32.441081 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da50b1d9-64ee-4daa-a722-653c33986bbb None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:32.445426 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da50b1d9-64ee-4daa-a722-653c33986bbb None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:32.446336 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da50b1d9-64ee-4daa-a722-653c33986bbb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:32.446846 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da50b1d9-64ee-4daa-a722-653c33986bbb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:32.448435 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da50b1d9-64ee-4daa-a722-653c33986bbb None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:32.469813 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 336/693] 10.222.0.102 () {56 vars in 1114 bytes} [Thu Sep 1 17:18:32 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:32.477707 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da50b1d9-64ee-4daa-a722-653c33986bbb None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:32.482600 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49f7f26-e31f-4139-8294-d1bd27099c7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:32.483364 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49f7f26-e31f-4139-8294-d1bd27099c7c None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:32.483647 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49f7f26-e31f-4139-8294-d1bd27099c7c None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:32.483918 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49f7f26-e31f-4139-8294-d1bd27099c7c None admin] PATH_INFO: `/v3/users/nova` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:32.495885 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49f7f26-e31f-4139-8294-d1bd27099c7c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:32.497354 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49f7f26-e31f-4139-8294-d1bd27099c7c None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:32.509210 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-a49f7f26-e31f-4139-8294-d1bd27099c7c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 17:18:32.509210 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:32.509210 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:32.509210 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:32.509210 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:32.509210 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:32.509210 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:32.509210 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:32.509210 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:32.509210 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:32.509210 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:32.509210 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:32.509210 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:18:32.509210 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:18:32.509210 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:18:32.509210 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:18:32.510177 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:32.510177 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:32.510177 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:18:32.510177 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:18:32.510177 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:18:32.510177 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:18:32.510177 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:32.510177 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:32.510177 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:32.510177 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:32.510177 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:32.510177 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:32.510177 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:32.510177 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:32.510177 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:32.510177 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:32.510177 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:32.510177 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:32.511127 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:32.511127 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:32.511127 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:18:32.511127 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:18:32.511127 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:18:32.511127 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:18:32.511127 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:18:32.511127 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:18:32.511127 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 17:18:32.511127 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:18:32.511127 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 358/694] 10.222.0.102 () {56 vars in 1098 bytes} [Thu Sep 1 17:18:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:18:32.516012 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49f7f26-e31f-4139-8294-d1bd27099c7c None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:32.609681 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:32.659751 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da50b1d9-64ee-4daa-a722-653c33986bbb None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:32.665766 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49f7f26-e31f-4139-8294-d1bd27099c7c None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:32.725199 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ca9e0f-1199-4bcc-8f12-122f863f68b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:32.725972 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ca9e0f-1199-4bcc-8f12-122f863f68b5 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:32.726257 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ca9e0f-1199-4bcc-8f12-122f863f68b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:32.726553 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ca9e0f-1199-4bcc-8f12-122f863f68b5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:32.741805 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ca9e0f-1199-4bcc-8f12-122f863f68b5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:32.743323 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ca9e0f-1199-4bcc-8f12-122f863f68b5 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:32.755907 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-d5ca9e0f-1199-4bcc-8f12-122f863f68b5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:18:32.755907 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:32.755907 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:32.755907 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:32.755907 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:32.755907 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:32.755907 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:32.755907 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:32.755907 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:32.755907 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:32.755907 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:32.755907 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:32.755907 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:18:32.755907 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:18:32.755907 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:18:32.756787 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:18:32.756787 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:32.756787 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:32.756787 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:32.756787 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:32.756787 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:32.756787 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:32.756787 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:32.756787 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:32.756787 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:32.756787 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:32.756787 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:32.756787 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:32.756787 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:32.756787 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:32.756787 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:32.756787 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:32.757816 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:18:32.757816 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:18:32.757816 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:18:32.757816 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:18:32.757816 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:18:32.757816 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:18:32.757816 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:18:32.757816 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:18:32.757816 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 337/695] 10.222.0.102 () {56 vars in 1116 bytes} [Thu Sep 1 17:18:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:18:32.758987 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7692fc98-cb4a-48f4-8f89-0a56607bf7df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:32.759757 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7692fc98-cb4a-48f4-8f89-0a56607bf7df None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:32.760154 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7692fc98-cb4a-48f4-8f89-0a56607bf7df None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:32.760488 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724a4b94-afc1-4538-8b21-7bb000eb0a85 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:32.760689 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7692fc98-cb4a-48f4-8f89-0a56607bf7df None admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:32.760780 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724a4b94-afc1-4538-8b21-7bb000eb0a85 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:32.761033 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724a4b94-afc1-4538-8b21-7bb000eb0a85 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:32.761786 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7692fc98-cb4a-48f4-8f89-0a56607bf7df None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:32.762238 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7692fc98-cb4a-48f4-8f89-0a56607bf7df None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:32.763794 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 338/696] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:18:32 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Sep 01 17:18:32.763906 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7692fc98-cb4a-48f4-8f89-0a56607bf7df None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:32.764499 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724a4b94-afc1-4538-8b21-7bb000eb0a85 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:32.784124 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 359/697] 10.222.0.102 () {56 vars in 1138 bytes} [Thu Sep 1 17:18:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 370 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:18:32.786228 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246b02e1-5c39-4fde-8f99-5d13eed6fa0c None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:32.786734 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246b02e1-5c39-4fde-8f99-5d13eed6fa0c None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:32.787222 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246b02e1-5c39-4fde-8f99-5d13eed6fa0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:32.806033 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-246b02e1-5c39-4fde-8f99-5d13eed6fa0c None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:18:32.807511 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-246b02e1-5c39-4fde-8f99-5d13eed6fa0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:18:32.839045 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:32.897599 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 360/698] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:18:32 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:18:32.899023 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246b02e1-5c39-4fde-8f99-5d13eed6fa0c None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:32.908830 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724a4b94-afc1-4538-8b21-7bb000eb0a85 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:32.948654 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac24b1b-096b-4c2d-88d1-81c579d09f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:32.949207 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac24b1b-096b-4c2d-88d1-81c579d09f9a None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:32.949513 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac24b1b-096b-4c2d-88d1-81c579d09f9a None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:32.949789 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac24b1b-096b-4c2d-88d1-81c579d09f9a None admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:32.950551 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac24b1b-096b-4c2d-88d1-81c579d09f9a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:32.950867 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac24b1b-096b-4c2d-88d1-81c579d09f9a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:32.952594 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac24b1b-096b-4c2d-88d1-81c579d09f9a None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:32.966594 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 339/699] 10.222.0.102 () {56 vars in 1153 bytes} [Thu Sep 1 17:18:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:32.978585 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac24b1b-096b-4c2d-88d1-81c579d09f9a None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:33.050841 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:33.091613 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac24b1b-096b-4c2d-88d1-81c579d09f9a None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:33.102211 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:33.130733 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf3412a-c625-438b-973f-10fc616dbd81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:33.131739 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf3412a-c625-438b-973f-10fc616dbd81 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:33.132287 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf3412a-c625-438b-973f-10fc616dbd81 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:33.132764 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf3412a-c625-438b-973f-10fc616dbd81 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:33.133858 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf3412a-c625-438b-973f-10fc616dbd81 None admin] RBAC: Adding query filter params (role.id=f21e23654e4740158c7d4f46e726e515, scope.project.id=e78fff7f1f6e4f68b8d233ad2e5bbd4f, user.id=d55edf6cd1804006aa8269b4c6f80aec) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:33.134408 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf3412a-c625-438b-973f-10fc616dbd81 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:33.136078 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf3412a-c625-438b-973f-10fc616dbd81 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:33.143480 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 340/700] 10.222.0.102 () {56 vars in 1379 bytes} [Thu Sep 1 17:18:32 2022] GET /identity/v3/role_assignments?user.id=d55edf6cd1804006aa8269b4c6f80aec&scope.project.id=e78fff7f1f6e4f68b8d233ad2e5bbd4f&role.id=f21e23654e4740158c7d4f46e726e515 => generated 610 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:33.155345 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246b02e1-5c39-4fde-8f99-5d13eed6fa0c None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:33.198594 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b76a089-3538-4d17-964d-c282572608b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:33.199722 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b76a089-3538-4d17-964d-c282572608b1 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:33.200215 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b76a089-3538-4d17-964d-c282572608b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:33.200686 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b76a089-3538-4d17-964d-c282572608b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:33.282173 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf3412a-c625-438b-973f-10fc616dbd81 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:33.283891 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b76a089-3538-4d17-964d-c282572608b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:33.285508 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b76a089-3538-4d17-964d-c282572608b1 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:33.393121 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:33.425514 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf3412a-c625-438b-973f-10fc616dbd81 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:33.459731 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 361/701] 10.222.0.102 () {58 vars in 1287 bytes} [Thu Sep 1 17:18:32 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 561 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 17:18:33.464921 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0e8ad4-c513-4ee0-b245-80538989e0a2 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:33.464921 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0e8ad4-c513-4ee0-b245-80538989e0a2 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:33.464921 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0e8ad4-c513-4ee0-b245-80538989e0a2 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:33.464921 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 362/702] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:18:33 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:18:33.466154 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0e8ad4-c513-4ee0-b245-80538989e0a2 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:33.475153 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6009697-4ab4-4c84-a426-5b1f23ab2876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:33.475793 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6009697-4ab4-4c84-a426-5b1f23ab2876 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:33.476119 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6009697-4ab4-4c84-a426-5b1f23ab2876 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:33.476410 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6009697-4ab4-4c84-a426-5b1f23ab2876 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:33.488284 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6009697-4ab4-4c84-a426-5b1f23ab2876 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:33.489767 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6009697-4ab4-4c84-a426-5b1f23ab2876 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:33.507904 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-e6009697-4ab4-4c84-a426-5b1f23ab2876 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:33.507904 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:33.507904 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:33.507904 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:33.507904 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:33.507904 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:33.507904 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:33.507904 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:33.507904 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:33.507904 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:33.507904 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:33.507904 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:33.507904 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:18:33.507904 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:18:33.507904 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:18:33.507904 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:18:33.508890 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:33.508890 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:33.508890 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:33.508890 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:33.508890 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:33.508890 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:33.508890 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:33.508890 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:33.508890 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:33.508890 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:33.508890 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:33.508890 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:33.508890 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:33.508890 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:33.508890 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:33.508890 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:33.508890 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:18:33.509899 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:18:33.509899 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:33.509899 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:18:33.509899 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 341/703] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:18:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:18:33.510586 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bc15f1-e07d-4bc6-8f8d-a1e5f05222a3 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:33.510861 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bc15f1-e07d-4bc6-8f8d-a1e5f05222a3 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:33.511135 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bc15f1-e07d-4bc6-8f8d-a1e5f05222a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:33.535866 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-49bc15f1-e07d-4bc6-8f8d-a1e5f05222a3 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:18:33.537324 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-49bc15f1-e07d-4bc6-8f8d-a1e5f05222a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:18:33.557155 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:33.618656 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 342/704] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:18:33 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:18:33.620340 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dbdabd-be44-43e9-978a-89c0a982d3de None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:33.620818 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dbdabd-be44-43e9-978a-89c0a982d3de None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:33.621278 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dbdabd-be44-43e9-978a-89c0a982d3de None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:33.623001 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 343/705] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:18:33 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:18:33.623917 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dbdabd-be44-43e9-978a-89c0a982d3de None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:33.697691 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0e8ad4-c513-4ee0-b245-80538989e0a2 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:33.734023 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355dd07b-ac37-4a4d-895c-b3918e94e46f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:33.734720 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355dd07b-ac37-4a4d-895c-b3918e94e46f None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:33.735090 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355dd07b-ac37-4a4d-895c-b3918e94e46f None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:33.735451 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355dd07b-ac37-4a4d-895c-b3918e94e46f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:33.748234 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355dd07b-ac37-4a4d-895c-b3918e94e46f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:33.749859 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355dd07b-ac37-4a4d-895c-b3918e94e46f None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:33.767056 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-355dd07b-ac37-4a4d-895c-b3918e94e46f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:33.767056 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:33.767056 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:33.767056 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:33.767056 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:33.767056 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:33.767056 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:33.767056 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:33.767056 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:33.767056 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:33.767056 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:33.767056 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:33.767056 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:18:33.767056 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:18:33.767056 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:18:33.767056 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:18:33.768318 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:33.768318 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:33.768318 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:33.768318 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:33.768318 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:33.768318 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:33.768318 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:33.768318 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:33.768318 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:33.768318 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:33.768318 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:33.768318 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:33.768318 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:33.768318 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:33.768318 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:33.768318 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:33.768318 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:18:33.770104 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:18:33.770104 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:33.770104 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:18:33.771327 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 363/706] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:18:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:33.772705 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9d3bd8-9d47-4ef9-96c7-4c938ff8838f None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:33.773084 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9d3bd8-9d47-4ef9-96c7-4c938ff8838f None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:33.773471 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9d3bd8-9d47-4ef9-96c7-4c938ff8838f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:33.775948 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dbdabd-be44-43e9-978a-89c0a982d3de None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:33.792796 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-8f9d3bd8-9d47-4ef9-96c7-4c938ff8838f None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:18:33.794186 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-8f9d3bd8-9d47-4ef9-96c7-4c938ff8838f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:18:33.796476 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:33.820936 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c8429b-5fcc-4d3f-aefe-d9777c22b674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:33.821885 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c8429b-5fcc-4d3f-aefe-d9777c22b674 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:33.822311 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c8429b-5fcc-4d3f-aefe-d9777c22b674 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:33.822729 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c8429b-5fcc-4d3f-aefe-d9777c22b674 None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:33.823529 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c8429b-5fcc-4d3f-aefe-d9777c22b674 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:33.823987 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c8429b-5fcc-4d3f-aefe-d9777c22b674 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:33.825701 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c8429b-5fcc-4d3f-aefe-d9777c22b674 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:33.839001 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 344/707] 10.222.0.102 () {56 vars in 1114 bytes} [Thu Sep 1 17:18:33 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:33.840103 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c8429b-5fcc-4d3f-aefe-d9777c22b674 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:33.858513 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:33.893347 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 364/708] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:18:33 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:18:33.893347 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9d3bd8-9d47-4ef9-96c7-4c938ff8838f None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:33.972095 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c8429b-5fcc-4d3f-aefe-d9777c22b674 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:34.009802 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f35d736-9d93-4f14-8708-d0534fd37280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:34.010335 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f35d736-9d93-4f14-8708-d0534fd37280 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:34.010888 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f35d736-9d93-4f14-8708-d0534fd37280 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:34.011176 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f35d736-9d93-4f14-8708-d0534fd37280 None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:34.011819 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f35d736-9d93-4f14-8708-d0534fd37280 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:34.012128 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f35d736-9d93-4f14-8708-d0534fd37280 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:34.014174 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f35d736-9d93-4f14-8708-d0534fd37280 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:34.019779 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9d3bd8-9d47-4ef9-96c7-4c938ff8838f None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:34.029367 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 345/709] 10.222.0.102 () {56 vars in 1114 bytes} [Thu Sep 1 17:18:33 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:34.030719 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f35d736-9d93-4f14-8708-d0534fd37280 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:34.063400 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2a5594-26f2-49b5-b4b2-e4e60e7f3a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:34.064495 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2a5594-26f2-49b5-b4b2-e4e60e7f3a87 None admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:34.065075 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2a5594-26f2-49b5-b4b2-e4e60e7f3a87 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:34.065572 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2a5594-26f2-49b5-b4b2-e4e60e7f3a87 None admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:34.066709 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2a5594-26f2-49b5-b4b2-e4e60e7f3a87 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:34.069036 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2a5594-26f2-49b5-b4b2-e4e60e7f3a87 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:34.082596 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:34.100493 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 365/710] 10.222.0.102 () {60 vars in 1137 bytes} [Thu Sep 1 17:18:33 2022] POST /identity/v3/users => generated 255 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:18:34.102371 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2a5594-26f2-49b5-b4b2-e4e60e7f3a87 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:34.127653 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:34.184709 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f35d736-9d93-4f14-8708-d0534fd37280 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:34.235126 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1817a1f0-c9c2-473a-b61c-65850dfbf678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:34.236197 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1817a1f0-c9c2-473a-b61c-65850dfbf678 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:34.236691 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1817a1f0-c9c2-473a-b61c-65850dfbf678 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:34.237198 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1817a1f0-c9c2-473a-b61c-65850dfbf678 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:34.238098 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1817a1f0-c9c2-473a-b61c-65850dfbf678 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:34.239735 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1817a1f0-c9c2-473a-b61c-65850dfbf678 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:34.252040 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-1817a1f0-c9c2-473a-b61c-65850dfbf678 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 17:18:34.252040 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:34.252040 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 17:18:34.252040 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 17:18:34.252040 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 17:18:34.252040 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 17:18:34.252040 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 17:18:34.252040 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:18:34.252040 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 17:18:34.252040 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:18:34.252040 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 17:18:34.252040 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:18:34.252040 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:34.252040 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:34.252040 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:34.252040 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:34.252040 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:34.253137 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:34.253137 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:34.253137 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:34.253137 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:34.253137 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:34.253137 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:34.253137 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 17:18:34.253137 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 17:18:34.253137 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 17:18:34.253137 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 17:18:34.253137 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:34.253137 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:34.253137 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:34.253137 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:34.253137 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:34.253137 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:34.254559 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:34.254559 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:34.254559 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:34.254559 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:34.254559 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:34.254559 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:34.254559 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:34.254559 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:34.254559 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:34.254559 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:34.254559 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 17:18:34.254559 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:18:34.254559 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 17:18:34.254559 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:18:34.262601 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 346/711] 10.222.0.102 () {56 vars in 1134 bytes} [Thu Sep 1 17:18:34 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:34.264014 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1817a1f0-c9c2-473a-b61c-65850dfbf678 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:34.283796 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2a5594-26f2-49b5-b4b2-e4e60e7f3a87 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:34.330815 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f2213d-06d9-4c49-9a4c-80e82790e416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:34.331463 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f2213d-06d9-4c49-9a4c-80e82790e416 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:34.332346 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f2213d-06d9-4c49-9a4c-80e82790e416 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:34.332570 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f2213d-06d9-4c49-9a4c-80e82790e416 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:34.342527 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f2213d-06d9-4c49-9a4c-80e82790e416 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:34.344544 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f2213d-06d9-4c49-9a4c-80e82790e416 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:34.359927 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-22f2213d-06d9-4c49-9a4c-80e82790e416 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:18:34.359927 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:34.359927 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:34.359927 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:34.359927 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:34.359927 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:34.359927 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:34.359927 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:34.359927 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:34.359927 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:34.359927 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:34.359927 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:34.359927 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:18:34.359927 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:18:34.359927 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:18:34.360849 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:18:34.360849 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:34.360849 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:34.360849 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:34.360849 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:34.360849 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:34.360849 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:34.360849 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:34.360849 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:34.360849 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:34.360849 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:34.360849 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:34.360849 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:34.360849 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:34.360849 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:34.360849 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:34.360849 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:34.361880 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:18:34.361880 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:18:34.361880 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:18:34.361880 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:18:34.361880 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:18:34.361880 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:18:34.361880 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:18:34.361880 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:18:34.361880 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 366/712] 10.222.0.102 () {56 vars in 1116 bytes} [Thu Sep 1 17:18:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:34.362376 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f2213d-06d9-4c49-9a4c-80e82790e416 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:34.393270 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1817a1f0-c9c2-473a-b61c-65850dfbf678 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:34.431515 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b69465b-02cd-427f-9ca8-d0d2d7bfd4fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:34.432426 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b69465b-02cd-427f-9ca8-d0d2d7bfd4fb None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:34.432848 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b69465b-02cd-427f-9ca8-d0d2d7bfd4fb None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:34.433449 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b69465b-02cd-427f-9ca8-d0d2d7bfd4fb None admin] PATH_INFO: `/v3/services/network` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:34.434167 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b69465b-02cd-427f-9ca8-d0d2d7bfd4fb None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:34.435786 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b69465b-02cd-427f-9ca8-d0d2d7bfd4fb None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:34.437486 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:34.448044 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-2b69465b-02cd-427f-9ca8-d0d2d7bfd4fb None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 17:18:34.448044 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:34.448044 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 17:18:34.448044 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 17:18:34.448044 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 17:18:34.448044 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 17:18:34.448044 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 17:18:34.448044 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:18:34.448044 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 17:18:34.448044 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:18:34.448044 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 17:18:34.448044 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:18:34.448044 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:34.448044 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:34.448044 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:34.448044 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:34.448044 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:34.449040 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:34.449040 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:34.449040 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:34.449040 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:34.449040 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:34.449040 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:34.449040 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 17:18:34.449040 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 17:18:34.449040 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 17:18:34.449040 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 17:18:34.449040 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:34.449040 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:34.449040 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:34.449040 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:34.449040 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:34.449040 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:34.449887 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:34.449887 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:34.449887 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:34.449887 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:34.449887 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:34.449887 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:34.449887 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:34.449887 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:34.449887 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:34.449887 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:34.449887 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 17:18:34.449887 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:18:34.449887 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 17:18:34.449887 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:18:34.452680 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 347/713] 10.222.0.102 () {56 vars in 1116 bytes} [Thu Sep 1 17:18:34 2022] GET /identity/v3/services/network => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:34.453706 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b69465b-02cd-427f-9ca8-d0d2d7bfd4fb None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:34.457951 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:34.514954 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f2213d-06d9-4c49-9a4c-80e82790e416 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:34.563159 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597f73b4-9b93-40b6-83ff-ce1f93a51ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:34.564046 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597f73b4-9b93-40b6-83ff-ce1f93a51ba3 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:34.564333 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597f73b4-9b93-40b6-83ff-ce1f93a51ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:34.564664 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597f73b4-9b93-40b6-83ff-ce1f93a51ba3 None admin] PATH_INFO: `/v3/services` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:34.565337 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597f73b4-9b93-40b6-83ff-ce1f93a51ba3 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:34.565632 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597f73b4-9b93-40b6-83ff-ce1f93a51ba3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:34.567857 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597f73b4-9b93-40b6-83ff-ce1f93a51ba3 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:34.570109 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b69465b-02cd-427f-9ca8-d0d2d7bfd4fb None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:34.573924 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 367/714] 10.222.0.102 () {56 vars in 1129 bytes} [Thu Sep 1 17:18:34 2022] GET /identity/v3/services?name=block-storage => generated 131 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:34.575915 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597f73b4-9b93-40b6-83ff-ce1f93a51ba3 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:34.626042 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812e4c07-5216-4a73-a87b-c3ece8666019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:34.626664 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812e4c07-5216-4a73-a87b-c3ece8666019 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:34.626948 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812e4c07-5216-4a73-a87b-c3ece8666019 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:34.627227 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812e4c07-5216-4a73-a87b-c3ece8666019 None admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:34.627926 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812e4c07-5216-4a73-a87b-c3ece8666019 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:34.628299 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812e4c07-5216-4a73-a87b-c3ece8666019 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:34.630368 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812e4c07-5216-4a73-a87b-c3ece8666019 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:34.652757 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 348/715] 10.222.0.102 () {56 vars in 1153 bytes} [Thu Sep 1 17:18:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:34.654030 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812e4c07-5216-4a73-a87b-c3ece8666019 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:34.707059 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597f73b4-9b93-40b6-83ff-ce1f93a51ba3 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:34.725550 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:34.733146 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:34.745818 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad862535-f8e7-4a9d-ac68-4e3113d36675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:34.746757 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad862535-f8e7-4a9d-ac68-4e3113d36675 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:34.747263 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad862535-f8e7-4a9d-ac68-4e3113d36675 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:34.747767 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad862535-f8e7-4a9d-ac68-4e3113d36675 None admin] PATH_INFO: `/v3/services` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:34.748580 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad862535-f8e7-4a9d-ac68-4e3113d36675 None admin] RBAC: Adding query filter params (name=network) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:34.749128 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad862535-f8e7-4a9d-ac68-4e3113d36675 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:34.750771 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad862535-f8e7-4a9d-ac68-4e3113d36675 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:34.756044 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 368/716] 10.222.0.102 () {56 vars in 1117 bytes} [Thu Sep 1 17:18:34 2022] GET /identity/v3/services?name=network => generated 125 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:34.757210 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad862535-f8e7-4a9d-ac68-4e3113d36675 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:34.790869 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812e4c07-5216-4a73-a87b-c3ece8666019 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:34.830145 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccee0a5-016b-40b5-b597-172fea12ed12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:34.831081 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccee0a5-016b-40b5-b597-172fea12ed12 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:34.831627 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccee0a5-016b-40b5-b597-172fea12ed12 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:34.832094 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccee0a5-016b-40b5-b597-172fea12ed12 None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:34.832907 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccee0a5-016b-40b5-b597-172fea12ed12 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:34.833465 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccee0a5-016b-40b5-b597-172fea12ed12 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:34.835040 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccee0a5-016b-40b5-b597-172fea12ed12 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:34.841773 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 349/717] 10.222.0.102 () {56 vars in 1129 bytes} [Thu Sep 1 17:18:34 2022] GET /identity/v3/services?type=block-storage => generated 131 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:34.842934 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccee0a5-016b-40b5-b597-172fea12ed12 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:34.932956 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad862535-f8e7-4a9d-ac68-4e3113d36675 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:35.011216 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f24816-b884-4bb6-b8a7-70077c39cf2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:35.012197 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f24816-b884-4bb6-b8a7-70077c39cf2e None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:35.012681 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f24816-b884-4bb6-b8a7-70077c39cf2e None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:35.013144 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f24816-b884-4bb6-b8a7-70077c39cf2e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:35.014084 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f24816-b884-4bb6-b8a7-70077c39cf2e None admin] RBAC: Adding query filter params (role.id=36577a6f46d642d4b81805ede9872490, scope.project.id=e78fff7f1f6e4f68b8d233ad2e5bbd4f, user.id=5f8293316baf4774b62fc11d55d50865) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:35.014645 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f24816-b884-4bb6-b8a7-70077c39cf2e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:35.016289 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f24816-b884-4bb6-b8a7-70077c39cf2e None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:35.023628 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 369/718] 10.222.0.102 () {56 vars in 1379 bytes} [Thu Sep 1 17:18:34 2022] GET /identity/v3/role_assignments?user.id=5f8293316baf4774b62fc11d55d50865&scope.project.id=e78fff7f1f6e4f68b8d233ad2e5bbd4f&role.id=36577a6f46d642d4b81805ede9872490 => generated 260 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:35.044798 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccee0a5-016b-40b5-b597-172fea12ed12 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:35.079403 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:35.092427 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce269ed-63c5-4ae8-afce-2c79a396e03b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:35.093479 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce269ed-63c5-4ae8-afce-2c79a396e03b None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:35.093984 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce269ed-63c5-4ae8-afce-2c79a396e03b None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:35.094448 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce269ed-63c5-4ae8-afce-2c79a396e03b None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:35.095347 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce269ed-63c5-4ae8-afce-2c79a396e03b None admin] RBAC: Adding query filter params (type=network) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:35.095871 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce269ed-63c5-4ae8-afce-2c79a396e03b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:35.098108 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce269ed-63c5-4ae8-afce-2c79a396e03b None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:35.104576 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 350/719] 10.222.0.102 () {56 vars in 1117 bytes} [Thu Sep 1 17:18:34 2022] GET /identity/v3/services?type=network => generated 125 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:35.395306 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1f4a3e-90b0-4f62-a477-16fc59f4d2c0 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:35.395653 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1f4a3e-90b0-4f62-a477-16fc59f4d2c0 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:35.395979 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1f4a3e-90b0-4f62-a477-16fc59f4d2c0 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:35.397611 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 370/720] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:18:35 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:18:35.402877 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f3bcba-6df6-43ff-945e-5b51cea35aec None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:35.403172 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f3bcba-6df6-43ff-945e-5b51cea35aec None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:35.403452 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f3bcba-6df6-43ff-945e-5b51cea35aec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:35.424005 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-41f3bcba-6df6-43ff-945e-5b51cea35aec None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:18:35.425739 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-41f3bcba-6df6-43ff-945e-5b51cea35aec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:18:35.507527 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 351/721] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:18:35 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:18:35.802406 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1f4a3e-90b0-4f62-a477-16fc59f4d2c0 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:35.874030 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:35.943006 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1f4a3e-90b0-4f62-a477-16fc59f4d2c0 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:35.987956 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fede7d0-436a-4577-9236-b49d205d8d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:35.988972 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fede7d0-436a-4577-9236-b49d205d8d3e None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:35.989486 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fede7d0-436a-4577-9236-b49d205d8d3e None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:35.989971 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fede7d0-436a-4577-9236-b49d205d8d3e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:36.006794 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fede7d0-436a-4577-9236-b49d205d8d3e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:18:36.011923 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-1fede7d0-436a-4577-9236-b49d205d8d3e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:18:36.011923 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:36.011923 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:36.011923 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:36.011923 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:36.011923 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:36.011923 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:36.011923 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:36.011923 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:36.011923 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:36.011923 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:36.011923 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:36.011923 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:18:36.011923 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:18:36.011923 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:18:36.011923 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:18:36.012809 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:18:36.012809 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:18:36.012809 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:18:36.012809 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:18:36.012809 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:36.012809 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:36.012809 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:36.012809 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:36.012809 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:36.012809 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:36.012809 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:36.012809 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:36.012809 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:36.012809 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:36.012809 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:36.012809 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:36.013665 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:36.013665 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:36.013665 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:36.013665 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:36.013665 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:18:36.013665 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:18:36.013665 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:18:36.013665 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:18:36.013665 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:18:36.013665 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:18:36.013665 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:18:36.013665 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:18:36.021068 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 371/722] 10.222.0.102 () {56 vars in 1101 bytes} [Thu Sep 1 17:18:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:36.022327 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f3bcba-6df6-43ff-945e-5b51cea35aec None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:36.169616 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f3bcba-6df6-43ff-945e-5b51cea35aec None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:36.193291 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:36.215135 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a732951-578d-4d0c-86ce-648f724f60af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:36.215799 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a732951-578d-4d0c-86ce-648f724f60af None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:36.216087 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a732951-578d-4d0c-86ce-648f724f60af None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:36.216365 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a732951-578d-4d0c-86ce-648f724f60af None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:36.217039 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a732951-578d-4d0c-86ce-648f724f60af None admin] RBAC: Adding query filter params (name=admin) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:36.217374 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a732951-578d-4d0c-86ce-648f724f60af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:36.218780 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a732951-578d-4d0c-86ce-648f724f60af None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:36.233798 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 352/723] 10.222.0.102 () {56 vars in 1104 bytes} [Thu Sep 1 17:18:36 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:36.240087 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fede7d0-436a-4577-9236-b49d205d8d3e None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:36.357238 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fede7d0-436a-4577-9236-b49d205d8d3e None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:36.370266 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:36.403270 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813056a2-9807-4b82-b66f-66d0301bc19b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:36.403476 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797b91af-67b2-4740-8cdf-f96b7d2318e2 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:36.403754 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797b91af-67b2-4740-8cdf-f96b7d2318e2 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:36.403922 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813056a2-9807-4b82-b66f-66d0301bc19b None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:36.404045 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797b91af-67b2-4740-8cdf-f96b7d2318e2 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:36.404209 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813056a2-9807-4b82-b66f-66d0301bc19b None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:36.404429 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813056a2-9807-4b82-b66f-66d0301bc19b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:36.410183 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 353/724] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:18:36 2022] GET /identity => generated 271 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:18:36.415746 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813056a2-9807-4b82-b66f-66d0301bc19b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:36.417197 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813056a2-9807-4b82-b66f-66d0301bc19b None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:36.418658 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9d4aa6-897b-4c26-afcb-dcc699021c0d None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:36.418946 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9d4aa6-897b-4c26-afcb-dcc699021c0d None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:36.419222 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9d4aa6-897b-4c26-afcb-dcc699021c0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:36.433727 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-813056a2-9807-4b82-b66f-66d0301bc19b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:36.433727 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:36.433727 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:36.433727 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:36.433727 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:36.433727 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:36.433727 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:36.433727 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:36.433727 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:36.433727 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:36.433727 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:36.433727 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:36.433727 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:18:36.433727 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:18:36.433727 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:18:36.433727 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:18:36.434766 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:36.434766 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:36.434766 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:36.434766 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:36.434766 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:36.434766 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:36.434766 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:36.434766 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:36.434766 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:36.434766 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:36.434766 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:36.434766 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:36.434766 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:36.434766 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:36.434766 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:36.434766 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:36.434766 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:18:36.435695 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:18:36.435695 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:36.435695 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:18:36.435695 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 372/725] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:18:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:36.440390 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813056a2-9807-4b82-b66f-66d0301bc19b None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:36.454494 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-1d9d4aa6-897b-4c26-afcb-dcc699021c0d None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:18:36.455700 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-1d9d4aa6-897b-4c26-afcb-dcc699021c0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:18:36.555984 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 354/726] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:18:36 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:18:36.694511 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813056a2-9807-4b82-b66f-66d0301bc19b None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:36.730423 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa36a848-66cb-49eb-a9e7-0b41301fa860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:36.731478 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa36a848-66cb-49eb-a9e7-0b41301fa860 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:36.731965 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa36a848-66cb-49eb-a9e7-0b41301fa860 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:36.732433 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa36a848-66cb-49eb-a9e7-0b41301fa860 None admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:36.737337 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa36a848-66cb-49eb-a9e7-0b41301fa860 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:36.737337 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa36a848-66cb-49eb-a9e7-0b41301fa860 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:36.737337 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa36a848-66cb-49eb-a9e7-0b41301fa860 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:36.747796 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 373/727] 10.222.0.102 () {56 vars in 1114 bytes} [Thu Sep 1 17:18:36 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:18:36.755753 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9d4aa6-897b-4c26-afcb-dcc699021c0d None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:36.878981 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa36a848-66cb-49eb-a9e7-0b41301fa860 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:36.950577 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:36.963153 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9d4aa6-897b-4c26-afcb-dcc699021c0d None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:36.980612 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:37.018608 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efee8bde-49b1-413b-9d3f-3403a7ac8c58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:37.019284 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efee8bde-49b1-413b-9d3f-3403a7ac8c58 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:37.019565 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efee8bde-49b1-413b-9d3f-3403a7ac8c58 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:37.019838 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efee8bde-49b1-413b-9d3f-3403a7ac8c58 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:37.030493 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efee8bde-49b1-413b-9d3f-3403a7ac8c58 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:37.031949 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efee8bde-49b1-413b-9d3f-3403a7ac8c58 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:37.053160 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-efee8bde-49b1-413b-9d3f-3403a7ac8c58 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 01 17:18:37.053160 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:37.053160 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:37.053160 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:37.053160 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:37.053160 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:37.053160 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:37.053160 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:37.053160 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:37.053160 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:37.053160 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:37.053160 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:37.053160 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:18:37.053160 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:18:37.053160 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:18:37.053160 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:18:37.054453 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:37.054453 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:37.054453 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:18:37.054453 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:18:37.054453 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:18:37.054453 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:18:37.054453 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:37.054453 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:37.054453 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:37.054453 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:37.054453 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:37.054453 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:37.054453 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:37.054453 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:37.054453 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:37.054453 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:37.054453 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:37.054453 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:37.055498 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:37.055498 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:37.055498 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:18:37.055498 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:18:37.055498 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:18:37.055498 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:18:37.055498 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:18:37.055498 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:18:37.055498 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 01 17:18:37.055498 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:18:37.056549 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 355/728] 10.222.0.102 () {56 vars in 1101 bytes} [Thu Sep 1 17:18:36 2022] GET /identity/v3/users/swift => generated 83 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:18:37.071008 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efee8bde-49b1-413b-9d3f-3403a7ac8c58 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:37.113892 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa36a848-66cb-49eb-a9e7-0b41301fa860 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:37.179002 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16718ff1-3bc9-4fe4-a517-1d783eca1578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:37.181163 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16718ff1-3bc9-4fe4-a517-1d783eca1578 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:37.182795 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16718ff1-3bc9-4fe4-a517-1d783eca1578 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:37.183713 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16718ff1-3bc9-4fe4-a517-1d783eca1578 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:37.206914 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16718ff1-3bc9-4fe4-a517-1d783eca1578 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:18:37.214461 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-16718ff1-3bc9-4fe4-a517-1d783eca1578 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:18:37.214461 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:37.214461 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:37.214461 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:37.214461 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:37.214461 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:37.214461 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:37.214461 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:37.214461 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:37.214461 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:37.214461 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:37.214461 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:37.214461 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:18:37.214461 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:18:37.214461 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:18:37.214461 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:18:37.215600 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:18:37.215600 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:18:37.215600 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:18:37.215600 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:18:37.215600 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:37.215600 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:37.215600 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:37.215600 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:37.215600 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:37.215600 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:37.215600 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:37.215600 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:37.215600 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:37.215600 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:37.215600 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:37.215600 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:37.216620 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:37.216620 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:37.216620 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:37.216620 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:37.216620 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:18:37.216620 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:18:37.216620 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:18:37.216620 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:18:37.216620 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:18:37.216620 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:18:37.216620 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:18:37.216620 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:18:37.225732 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 374/729] 10.222.0.102 () {56 vars in 1107 bytes} [Thu Sep 1 17:18:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:37.230101 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16718ff1-3bc9-4fe4-a517-1d783eca1578 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:37.364190 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:37.383417 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16718ff1-3bc9-4fe4-a517-1d783eca1578 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:37.402343 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efee8bde-49b1-413b-9d3f-3403a7ac8c58 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:37.438294 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dd0f21-fbfb-4a35-bb48-b033c5713505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:37.438932 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dd0f21-fbfb-4a35-bb48-b033c5713505 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:37.439210 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dd0f21-fbfb-4a35-bb48-b033c5713505 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:37.439480 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dd0f21-fbfb-4a35-bb48-b033c5713505 None admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:37.440208 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dd0f21-fbfb-4a35-bb48-b033c5713505 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:37.440513 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dd0f21-fbfb-4a35-bb48-b033c5713505 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:37.440620 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803025e8-74dd-4173-b830-6b652d5d5cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:37.441793 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803025e8-74dd-4173-b830-6b652d5d5cd5 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:37.442070 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803025e8-74dd-4173-b830-6b652d5d5cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:37.442338 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803025e8-74dd-4173-b830-6b652d5d5cd5 None admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:37.442551 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dd0f21-fbfb-4a35-bb48-b033c5713505 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:37.442991 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803025e8-74dd-4173-b830-6b652d5d5cd5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:37.443291 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803025e8-74dd-4173-b830-6b652d5d5cd5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:37.444943 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803025e8-74dd-4173-b830-6b652d5d5cd5 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:37.456504 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 375/730] 10.222.0.102 () {56 vars in 1108 bytes} [Thu Sep 1 17:18:37 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:37.458973 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ef4d5c-ba24-44b7-951b-4866f24d7f67 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:37.459204 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ef4d5c-ba24-44b7-951b-4866f24d7f67 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:37.459768 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ef4d5c-ba24-44b7-951b-4866f24d7f67 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:37.462157 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 356/732] 10.222.0.102 () {56 vars in 1140 bytes} [Thu Sep 1 17:18:37 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 372 bytes in 391 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:37.463257 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dd0f21-fbfb-4a35-bb48-b033c5713505 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:37.465181 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 376/732] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:18:37 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:18:37.468150 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b8a106-d7fc-4db1-9399-a310e384497e None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:37.468441 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b8a106-d7fc-4db1-9399-a310e384497e None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:37.468718 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b8a106-d7fc-4db1-9399-a310e384497e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:37.491925 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-96b8a106-d7fc-4db1-9399-a310e384497e None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:18:37.493788 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-96b8a106-d7fc-4db1-9399-a310e384497e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:18:37.495564 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:37.572990 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dd0f21-fbfb-4a35-bb48-b033c5713505 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:37.576865 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 377/733] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:18:37 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:18:37.577818 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b8a106-d7fc-4db1-9399-a310e384497e None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:37.614937 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c617c877-965f-454d-abed-76824e6046b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:37.615568 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c617c877-965f-454d-abed-76824e6046b9 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:37.615905 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c617c877-965f-454d-abed-76824e6046b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:37.616178 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c617c877-965f-454d-abed-76824e6046b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:37.626815 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c617c877-965f-454d-abed-76824e6046b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:37.628909 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c617c877-965f-454d-abed-76824e6046b9 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:37.643894 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-c617c877-965f-454d-abed-76824e6046b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:37.643894 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:37.643894 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:37.643894 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:37.643894 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:37.643894 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:37.643894 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:37.643894 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:37.643894 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:37.643894 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:37.643894 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:37.643894 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:37.643894 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:18:37.643894 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:18:37.643894 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:18:37.643894 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:18:37.644854 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:37.644854 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:37.644854 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:37.644854 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:37.644854 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:37.644854 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:37.644854 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:37.644854 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:37.644854 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:37.644854 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:37.644854 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:37.644854 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:37.644854 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:37.644854 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:37.644854 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:37.644854 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:37.644854 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:18:37.645763 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:18:37.645763 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:37.645763 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:18:37.649932 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 357/734] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:18:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:37.651221 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d842b4-3be1-487f-a4b4-c2ce2814c9f5 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:37.651508 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d842b4-3be1-487f-a4b4-c2ce2814c9f5 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:37.651784 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d842b4-3be1-487f-a4b4-c2ce2814c9f5 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:37.652383 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:37.653290 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 358/735] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:18:37 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:18:37.654051 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d842b4-3be1-487f-a4b4-c2ce2814c9f5 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:37.686396 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b8a106-d7fc-4db1-9399-a310e384497e None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:37.727764 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc24b3a-ef6b-4541-896a-36521b435450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:37.728831 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc24b3a-ef6b-4541-896a-36521b435450 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:37.729399 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc24b3a-ef6b-4541-896a-36521b435450 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:37.729677 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc24b3a-ef6b-4541-896a-36521b435450 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:37.757594 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:37.779566 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d842b4-3be1-487f-a4b4-c2ce2814c9f5 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:37.812124 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc24b3a-ef6b-4541-896a-36521b435450 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:37.813684 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc24b3a-ef6b-4541-896a-36521b435450 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:37.817139 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0499d0-bfd3-47c7-ba67-594f76b21fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:37.817775 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0499d0-bfd3-47c7-ba67-594f76b21fdd None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:37.818060 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0499d0-bfd3-47c7-ba67-594f76b21fdd None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:37.818329 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0499d0-bfd3-47c7-ba67-594f76b21fdd None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:37.818966 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0499d0-bfd3-47c7-ba67-594f76b21fdd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:37.819270 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0499d0-bfd3-47c7-ba67-594f76b21fdd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:37.820842 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0499d0-bfd3-47c7-ba67-594f76b21fdd None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:37.832870 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 359/736] 10.222.0.102 () {56 vars in 1114 bytes} [Thu Sep 1 17:18:37 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:37.834093 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d44a57d-4275-4f87-9709-0d8804601f7a None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:37.834384 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d44a57d-4275-4f87-9709-0d8804601f7a None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:37.834655 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d44a57d-4275-4f87-9709-0d8804601f7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:37.855667 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-5d44a57d-4275-4f87-9709-0d8804601f7a None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:18:37.857113 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-5d44a57d-4275-4f87-9709-0d8804601f7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:18:37.897702 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:37.926100 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 360/737] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:18:37 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:18:37.927362 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5a6b43-94f3-4954-957f-0fbee44f8602 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:37.927651 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5a6b43-94f3-4954-957f-0fbee44f8602 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:37.927922 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5a6b43-94f3-4954-957f-0fbee44f8602 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:37.929325 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 361/738] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:18:37 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:18:37.929884 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5a6b43-94f3-4954-957f-0fbee44f8602 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:37.935513 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 378/739] 10.222.0.102 () {58 vars in 1287 bytes} [Thu Sep 1 17:18:37 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 17:18:37.935513 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc24b3a-ef6b-4541-896a-36521b435450 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:38.006474 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:38.038005 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc24b3a-ef6b-4541-896a-36521b435450 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:38.041269 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5a6b43-94f3-4954-957f-0fbee44f8602 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:38.073021 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885db1b4-8f1e-49bc-aade-89579592070e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:38.073796 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885db1b4-8f1e-49bc-aade-89579592070e None admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:38.074078 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885db1b4-8f1e-49bc-aade-89579592070e None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:38.074344 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885db1b4-8f1e-49bc-aade-89579592070e None admin] PATH_INFO: `/v3/services` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:38.074995 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885db1b4-8f1e-49bc-aade-89579592070e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:38.076467 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885db1b4-8f1e-49bc-aade-89579592070e None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:38.076733 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b7b4c8-0278-40c8-8d91-a22e0a38fa67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:38.077387 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b7b4c8-0278-40c8-8d91-a22e0a38fa67 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:38.077669 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b7b4c8-0278-40c8-8d91-a22e0a38fa67 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:38.077938 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b7b4c8-0278-40c8-8d91-a22e0a38fa67 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:38.086929 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b7b4c8-0278-40c8-8d91-a22e0a38fa67 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:38.088549 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b7b4c8-0278-40c8-8d91-a22e0a38fa67 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:38.094644 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 379/740] 10.222.0.102 () {60 vars in 1146 bytes} [Thu Sep 1 17:18:37 2022] POST /identity/v3/services => generated 252 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:18:38.095261 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885db1b4-8f1e-49bc-aade-89579592070e None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:38.106629 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-45b7b4c8-0278-40c8-8d91-a22e0a38fa67 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 17:18:38.106629 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:38.106629 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:38.106629 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:38.106629 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:38.106629 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:38.106629 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:38.106629 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:38.106629 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:38.106629 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:38.106629 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:38.106629 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:38.106629 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:18:38.106629 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:18:38.106629 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:18:38.106629 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:18:38.107699 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:38.107699 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:38.107699 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:18:38.107699 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:18:38.107699 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:18:38.107699 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:18:38.107699 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:38.107699 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:38.107699 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:38.107699 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:38.107699 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:38.107699 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:38.107699 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:38.107699 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:38.107699 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:38.107699 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:38.107699 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:38.107699 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:38.108680 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:38.108680 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:38.108680 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:18:38.108680 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:18:38.108680 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:18:38.108680 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:18:38.108680 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:18:38.108680 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:18:38.108680 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 17:18:38.108680 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:18:38.116199 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 362/741] 10.222.0.102 () {56 vars in 1122 bytes} [Thu Sep 1 17:18:37 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:38.116199 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20ea805-a683-4d00-b010-8e730d44dfe8 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:38.116199 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20ea805-a683-4d00-b010-8e730d44dfe8 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:38.116199 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20ea805-a683-4d00-b010-8e730d44dfe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:38.138889 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-a20ea805-a683-4d00-b010-8e730d44dfe8 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:18:38.140104 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-a20ea805-a683-4d00-b010-8e730d44dfe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:18:38.143562 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:38.237353 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 363/742] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:18:38 2022] POST /identity/v3/auth/tokens => generated 904 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:18:38.237353 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20ea805-a683-4d00-b010-8e730d44dfe8 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:38.249019 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885db1b4-8f1e-49bc-aade-89579592070e None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:38.290763 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa64b128-4e6d-4b9f-a6db-1441b5f0030d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:38.291327 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa64b128-4e6d-4b9f-a6db-1441b5f0030d None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:38.291611 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa64b128-4e6d-4b9f-a6db-1441b5f0030d None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:38.291889 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa64b128-4e6d-4b9f-a6db-1441b5f0030d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:38.302428 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:38.304934 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa64b128-4e6d-4b9f-a6db-1441b5f0030d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:38.306538 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa64b128-4e6d-4b9f-a6db-1441b5f0030d None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:38.324614 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-fa64b128-4e6d-4b9f-a6db-1441b5f0030d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:38.324614 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:38.324614 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:38.324614 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:38.324614 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:38.324614 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:38.324614 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:38.324614 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:38.324614 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:38.324614 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:38.324614 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:38.324614 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:38.324614 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:18:38.324614 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:18:38.324614 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:18:38.324614 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:18:38.325674 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:38.325674 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:38.325674 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:38.325674 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:38.325674 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:38.325674 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:38.325674 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:38.325674 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:38.325674 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:38.325674 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:38.325674 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:38.325674 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:38.325674 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:38.325674 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:38.325674 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:38.325674 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:38.325674 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:18:38.326571 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:18:38.326571 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:38.326571 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:18:38.329452 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 380/743] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:18:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:38.330407 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa64b128-4e6d-4b9f-a6db-1441b5f0030d None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:38.366698 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20ea805-a683-4d00-b010-8e730d44dfe8 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:38.411502 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1ecd20-5727-4a2d-8ea8-9d83fe3b4109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:38.412632 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1ecd20-5727-4a2d-8ea8-9d83fe3b4109 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:38.413733 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1ecd20-5727-4a2d-8ea8-9d83fe3b4109 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:38.414018 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1ecd20-5727-4a2d-8ea8-9d83fe3b4109 None admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:38.414765 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1ecd20-5727-4a2d-8ea8-9d83fe3b4109 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:38.415097 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1ecd20-5727-4a2d-8ea8-9d83fe3b4109 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:38.416527 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1ecd20-5727-4a2d-8ea8-9d83fe3b4109 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:38.431834 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:38.439879 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 364/744] 10.222.0.102 () {56 vars in 1154 bytes} [Thu Sep 1 17:18:38 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 386 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:38.441657 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1ecd20-5727-4a2d-8ea8-9d83fe3b4109 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:38.455497 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa64b128-4e6d-4b9f-a6db-1441b5f0030d None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:38.499457 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c9dd5b-4b7a-4fd4-8f17-6676df062476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:38.500091 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c9dd5b-4b7a-4fd4-8f17-6676df062476 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:38.500371 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c9dd5b-4b7a-4fd4-8f17-6676df062476 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:38.500647 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c9dd5b-4b7a-4fd4-8f17-6676df062476 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:38.514185 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c9dd5b-4b7a-4fd4-8f17-6676df062476 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:18:38.519541 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-75c9dd5b-4b7a-4fd4-8f17-6676df062476 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:18:38.519541 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:38.519541 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:38.519541 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:38.519541 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:38.519541 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:38.519541 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:38.519541 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:38.519541 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:38.519541 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:38.519541 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:38.519541 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:38.519541 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:18:38.519541 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:18:38.519541 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:18:38.519541 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:18:38.520453 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:18:38.520453 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:18:38.520453 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:18:38.520453 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:18:38.520453 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:38.520453 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:38.520453 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:38.520453 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:38.520453 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:38.520453 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:38.520453 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:38.520453 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:38.520453 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:38.520453 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:38.520453 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:38.520453 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:38.521280 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:38.521280 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:38.521280 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:38.521280 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:38.521280 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:18:38.521280 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:18:38.521280 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:18:38.521280 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:18:38.521280 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:18:38.521280 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:18:38.521280 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:18:38.521280 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:18:38.521280 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 381/745] 10.222.0.102 () {56 vars in 1101 bytes} [Thu Sep 1 17:18:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:38.521986 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c9dd5b-4b7a-4fd4-8f17-6676df062476 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:38.553824 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1ecd20-5727-4a2d-8ea8-9d83fe3b4109 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:38.570847 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:38.590706 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99ae40d-2a98-49e1-9da4-ac4bc9d67b25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:38.591314 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99ae40d-2a98-49e1-9da4-ac4bc9d67b25 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:38.591589 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99ae40d-2a98-49e1-9da4-ac4bc9d67b25 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:38.591860 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99ae40d-2a98-49e1-9da4-ac4bc9d67b25 None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:38.592495 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99ae40d-2a98-49e1-9da4-ac4bc9d67b25 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:38.592801 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99ae40d-2a98-49e1-9da4-ac4bc9d67b25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:38.594223 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99ae40d-2a98-49e1-9da4-ac4bc9d67b25 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:38.608565 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 365/746] 10.222.0.102 () {56 vars in 1114 bytes} [Thu Sep 1 17:18:38 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:38.609389 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99ae40d-2a98-49e1-9da4-ac4bc9d67b25 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:38.638205 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c9dd5b-4b7a-4fd4-8f17-6676df062476 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:38.681996 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dac523-8364-4403-bdd8-695f5a0c1bce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:38.682601 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dac523-8364-4403-bdd8-695f5a0c1bce None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:38.682875 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dac523-8364-4403-bdd8-695f5a0c1bce None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:38.683139 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dac523-8364-4403-bdd8-695f5a0c1bce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:38.697080 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:38.717613 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99ae40d-2a98-49e1-9da4-ac4bc9d67b25 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:38.753672 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7364d1cd-8081-4297-9ecc-33f1e994bdc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:38.754296 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7364d1cd-8081-4297-9ecc-33f1e994bdc1 None admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:38.754576 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7364d1cd-8081-4297-9ecc-33f1e994bdc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:38.754846 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7364d1cd-8081-4297-9ecc-33f1e994bdc1 None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:38.755505 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7364d1cd-8081-4297-9ecc-33f1e994bdc1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:38.756891 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7364d1cd-8081-4297-9ecc-33f1e994bdc1 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:38.759199 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dac523-8364-4403-bdd8-695f5a0c1bce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:38.760998 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dac523-8364-4403-bdd8-695f5a0c1bce None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:38.775093 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 366/747] 10.222.0.102 () {60 vars in 1146 bytes} [Thu Sep 1 17:18:38 2022] POST /identity/v3/services => generated 241 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:18:38.776784 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7364d1cd-8081-4297-9ecc-33f1e994bdc1 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:38.840022 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:38.913540 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 382/748] 10.222.0.102 () {58 vars in 1287 bytes} [Thu Sep 1 17:18:38 2022] GET /identity/v3/auth/tokens => generated 904 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 17:18:38.913540 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dac523-8364-4403-bdd8-695f5a0c1bce None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:38.943493 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7364d1cd-8081-4297-9ecc-33f1e994bdc1 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:38.963546 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:38.990602 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deff73f8-5507-4c43-8bf1-9b9b904beee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:38.991426 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deff73f8-5507-4c43-8bf1-9b9b904beee2 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:38.991710 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deff73f8-5507-4c43-8bf1-9b9b904beee2 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:38.991987 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deff73f8-5507-4c43-8bf1-9b9b904beee2 None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:38.993138 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deff73f8-5507-4c43-8bf1-9b9b904beee2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:38.993532 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deff73f8-5507-4c43-8bf1-9b9b904beee2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:38.994929 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deff73f8-5507-4c43-8bf1-9b9b904beee2 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:39.015745 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 367/749] 10.222.0.102 () {56 vars in 1104 bytes} [Thu Sep 1 17:18:38 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:39.015745 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deff73f8-5507-4c43-8bf1-9b9b904beee2 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:39.034612 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dac523-8364-4403-bdd8-695f5a0c1bce None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:39.078261 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980c87b5-5687-4f38-b887-f576efbeaf7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:39.079369 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980c87b5-5687-4f38-b887-f576efbeaf7a None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:39.080172 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980c87b5-5687-4f38-b887-f576efbeaf7a None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:39.080942 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980c87b5-5687-4f38-b887-f576efbeaf7a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:39.089687 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980c87b5-5687-4f38-b887-f576efbeaf7a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:39.091550 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980c87b5-5687-4f38-b887-f576efbeaf7a None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:39.108094 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-980c87b5-5687-4f38-b887-f576efbeaf7a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:18:39.108094 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:39.108094 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:39.108094 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:39.108094 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:39.108094 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:39.108094 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:39.108094 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:39.108094 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:39.108094 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:39.108094 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:39.108094 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:39.108094 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:18:39.108094 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:18:39.108094 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:18:39.108987 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:18:39.108987 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:39.108987 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:39.108987 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:39.108987 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:39.108987 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:39.108987 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:39.108987 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:39.108987 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:39.108987 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:39.108987 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:39.108987 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:39.108987 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:39.108987 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:39.108987 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:39.108987 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:39.108987 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:39.109904 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:18:39.109904 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:18:39.109904 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:18:39.109904 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:18:39.109904 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:18:39.109904 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:18:39.109904 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:18:39.109904 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:18:39.113171 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 383/750] 10.222.0.102 () {56 vars in 1116 bytes} [Thu Sep 1 17:18:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:39.114460 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980c87b5-5687-4f38-b887-f576efbeaf7a None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:39.125912 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:39.135325 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deff73f8-5507-4c43-8bf1-9b9b904beee2 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:39.174144 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86179770-c164-49e6-932b-199aa216c78c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:39.174941 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86179770-c164-49e6-932b-199aa216c78c None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:39.175837 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86179770-c164-49e6-932b-199aa216c78c None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:39.176208 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86179770-c164-49e6-932b-199aa216c78c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:39.194638 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86179770-c164-49e6-932b-199aa216c78c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:39.196728 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86179770-c164-49e6-932b-199aa216c78c None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:39.216056 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-86179770-c164-49e6-932b-199aa216c78c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:39.216056 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:39.216056 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:39.216056 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:39.216056 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:39.216056 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:39.216056 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:39.216056 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:39.216056 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:39.216056 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:39.216056 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:39.216056 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:39.216056 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:18:39.216056 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:18:39.216056 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:18:39.216056 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:18:39.217072 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:39.217072 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:39.217072 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:39.217072 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:39.217072 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:39.217072 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:39.217072 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:39.217072 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:39.217072 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:39.217072 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:39.217072 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:39.217072 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:39.217072 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:39.217072 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:39.217072 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:39.217072 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:39.217072 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:18:39.218013 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:18:39.218013 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:39.218013 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:18:39.221440 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 368/751] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:18:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:39.222888 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86179770-c164-49e6-932b-199aa216c78c None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:39.241714 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980c87b5-5687-4f38-b887-f576efbeaf7a None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:39.243387 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:39.281654 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98d79f4-c2bc-47c5-8b5d-10b0d00780f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:39.282539 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98d79f4-c2bc-47c5-8b5d-10b0d00780f9 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:39.282842 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98d79f4-c2bc-47c5-8b5d-10b0d00780f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:39.283207 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98d79f4-c2bc-47c5-8b5d-10b0d00780f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:39.295720 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98d79f4-c2bc-47c5-8b5d-10b0d00780f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:39.297407 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98d79f4-c2bc-47c5-8b5d-10b0d00780f9 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:39.313921 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-e98d79f4-c2bc-47c5-8b5d-10b0d00780f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:39.313921 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:39.313921 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:39.313921 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:39.313921 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:39.313921 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:39.313921 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:39.313921 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:39.313921 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:39.313921 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:39.313921 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:39.313921 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:39.313921 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:18:39.313921 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:18:39.313921 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:18:39.313921 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:18:39.315140 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:39.315140 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:39.315140 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:39.315140 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:39.315140 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:39.315140 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:39.315140 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:39.315140 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:39.315140 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:39.315140 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:39.315140 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:39.315140 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:39.315140 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:39.315140 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:39.315140 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:39.315140 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:39.315140 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:18:39.316790 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:18:39.316790 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:39.316790 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:18:39.322472 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 384/752] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:18:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:39.323429 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98d79f4-c2bc-47c5-8b5d-10b0d00780f9 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:39.342553 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86179770-c164-49e6-932b-199aa216c78c None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:39.391740 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293a6cb5-4078-4c70-80d6-ffd21e59c325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:39.392628 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293a6cb5-4078-4c70-80d6-ffd21e59c325 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:39.393401 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293a6cb5-4078-4c70-80d6-ffd21e59c325 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:39.393901 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293a6cb5-4078-4c70-80d6-ffd21e59c325 None admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:39.395317 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293a6cb5-4078-4c70-80d6-ffd21e59c325 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:39.395800 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293a6cb5-4078-4c70-80d6-ffd21e59c325 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:39.398096 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293a6cb5-4078-4c70-80d6-ffd21e59c325 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:39.420486 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 369/753] 10.222.0.102 () {56 vars in 1153 bytes} [Thu Sep 1 17:18:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:39.422024 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293a6cb5-4078-4c70-80d6-ffd21e59c325 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:39.433911 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98d79f4-c2bc-47c5-8b5d-10b0d00780f9 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:39.435000 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:39.470050 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bece46-1de0-4508-9291-50b2782a3636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:39.470665 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bece46-1de0-4508-9291-50b2782a3636 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:39.470942 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bece46-1de0-4508-9291-50b2782a3636 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:39.471212 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bece46-1de0-4508-9291-50b2782a3636 None admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:39.471855 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bece46-1de0-4508-9291-50b2782a3636 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:39.472676 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bece46-1de0-4508-9291-50b2782a3636 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:39.474072 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bece46-1de0-4508-9291-50b2782a3636 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:39.491217 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 385/754] 10.222.0.102 () {56 vars in 1114 bytes} [Thu Sep 1 17:18:39 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:39.491217 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bece46-1de0-4508-9291-50b2782a3636 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:39.505069 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:39.573407 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293a6cb5-4078-4c70-80d6-ffd21e59c325 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:39.594686 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bece46-1de0-4508-9291-50b2782a3636 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:39.621539 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf6414c-1011-4785-9cce-3715df471057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:39.622733 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf6414c-1011-4785-9cce-3715df471057 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:39.623661 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf6414c-1011-4785-9cce-3715df471057 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:39.624409 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf6414c-1011-4785-9cce-3715df471057 None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:39.625610 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf6414c-1011-4785-9cce-3715df471057 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:39.626555 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf6414c-1011-4785-9cce-3715df471057 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:39.628633 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf6414c-1011-4785-9cce-3715df471057 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:39.631047 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5c4f66-7036-45ee-8dd8-ceca6e99cfc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:39.631640 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5c4f66-7036-45ee-8dd8-ceca6e99cfc8 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:39.631903 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5c4f66-7036-45ee-8dd8-ceca6e99cfc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:39.632295 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5c4f66-7036-45ee-8dd8-ceca6e99cfc8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:39.632883 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5c4f66-7036-45ee-8dd8-ceca6e99cfc8 None admin] RBAC: Adding query filter params (role.id=36577a6f46d642d4b81805ede9872490, scope.project.id=e78fff7f1f6e4f68b8d233ad2e5bbd4f, user.id=d55edf6cd1804006aa8269b4c6f80aec) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:39.633188 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5c4f66-7036-45ee-8dd8-ceca6e99cfc8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:39.634820 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5c4f66-7036-45ee-8dd8-ceca6e99cfc8 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:39.645335 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 386/755] 10.222.0.102 () {56 vars in 1379 bytes} [Thu Sep 1 17:18:39 2022] GET /identity/v3/role_assignments?user.id=d55edf6cd1804006aa8269b4c6f80aec&scope.project.id=e78fff7f1f6e4f68b8d233ad2e5bbd4f&role.id=36577a6f46d642d4b81805ede9872490 => generated 260 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:39.645335 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5c4f66-7036-45ee-8dd8-ceca6e99cfc8 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:39.657070 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 370/756] 10.222.0.102 () {56 vars in 1114 bytes} [Thu Sep 1 17:18:39 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:39.663606 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf6414c-1011-4785-9cce-3715df471057 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:39.766118 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:39.821381 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf6414c-1011-4785-9cce-3715df471057 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:39.840662 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:39.861586 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5c0840-558d-4582-a129-c0e7a59ace2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:39.862430 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5c0840-558d-4582-a129-c0e7a59ace2f None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:39.862750 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5c0840-558d-4582-a129-c0e7a59ace2f None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:39.863026 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5c0840-558d-4582-a129-c0e7a59ace2f None admin] PATH_INFO: `/v3/users/nova` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:39.872924 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5c0840-558d-4582-a129-c0e7a59ace2f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:39.875174 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5c0840-558d-4582-a129-c0e7a59ace2f None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:39.884180 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5c4f66-7036-45ee-8dd8-ceca6e99cfc8 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:39.888051 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-bd5c0840-558d-4582-a129-c0e7a59ace2f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 17:18:39.888051 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:39.888051 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:39.888051 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:39.888051 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:39.888051 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:39.888051 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:39.888051 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:39.888051 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:39.888051 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:39.888051 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:39.888051 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:39.888051 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:18:39.888051 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:18:39.888051 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:18:39.888051 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:18:39.888955 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:39.888955 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:39.888955 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:18:39.888955 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:18:39.888955 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:18:39.888955 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:18:39.888955 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:39.888955 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:39.888955 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:39.888955 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:39.888955 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:39.888955 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:39.888955 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:39.888955 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:39.888955 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:39.888955 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:39.888955 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:39.888955 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:39.889922 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:39.889922 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:39.889922 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:18:39.889922 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:18:39.889922 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:18:39.889922 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:18:39.889922 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:18:39.889922 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:18:39.889922 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 17:18:39.889922 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:18:39.889922 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 371/757] 10.222.0.102 () {56 vars in 1098 bytes} [Thu Sep 1 17:18:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:18:39.895585 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5c0840-558d-4582-a129-c0e7a59ace2f None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:39.967792 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4480812-65ee-4108-a8a2-8802938b619e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:39.968493 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4480812-65ee-4108-a8a2-8802938b619e None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:39.968705 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4480812-65ee-4108-a8a2-8802938b619e None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:39.968979 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4480812-65ee-4108-a8a2-8802938b619e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:39.986704 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4480812-65ee-4108-a8a2-8802938b619e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:39.988253 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4480812-65ee-4108-a8a2-8802938b619e None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:40.008637 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5c0840-558d-4582-a129-c0e7a59ace2f None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:40.010760 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-f4480812-65ee-4108-a8a2-8802938b619e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:18:40.010760 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:40.010760 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:40.010760 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:40.010760 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:40.010760 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:40.010760 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:40.010760 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:40.010760 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:40.010760 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:40.010760 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:40.010760 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:40.010760 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:18:40.010760 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:18:40.010760 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:18:40.011615 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:18:40.011615 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:40.011615 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:40.011615 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:40.011615 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:40.011615 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:40.011615 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:40.011615 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:40.011615 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:40.011615 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:40.011615 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:40.011615 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:40.011615 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:40.011615 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:40.011615 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:40.011615 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:40.011615 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:40.012483 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:18:40.012483 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:18:40.012483 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:18:40.012483 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:18:40.012483 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:18:40.012483 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:18:40.012483 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:18:40.012483 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:18:40.017433 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 387/758] 10.222.0.102 () {56 vars in 1116 bytes} [Thu Sep 1 17:18:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 371 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:40.019297 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4480812-65ee-4108-a8a2-8802938b619e None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:40.041397 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:40.053630 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64eed3e-ba2b-4c7b-b140-b127848fcd75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:40.054260 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64eed3e-ba2b-4c7b-b140-b127848fcd75 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:40.054523 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64eed3e-ba2b-4c7b-b140-b127848fcd75 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:40.054801 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64eed3e-ba2b-4c7b-b140-b127848fcd75 None admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:40.055532 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64eed3e-ba2b-4c7b-b140-b127848fcd75 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:40.055837 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64eed3e-ba2b-4c7b-b140-b127848fcd75 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:40.057351 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64eed3e-ba2b-4c7b-b140-b127848fcd75 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:40.077291 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 372/759] 10.222.0.102 () {56 vars in 1138 bytes} [Thu Sep 1 17:18:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 370 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:18:40.082622 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64eed3e-ba2b-4c7b-b140-b127848fcd75 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:40.193838 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64eed3e-ba2b-4c7b-b140-b127848fcd75 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:40.207318 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4480812-65ee-4108-a8a2-8802938b619e None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:40.231456 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b96e1d-ae2d-442d-9fdc-b72172bf4409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:40.232097 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b96e1d-ae2d-442d-9fdc-b72172bf4409 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:40.232377 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b96e1d-ae2d-442d-9fdc-b72172bf4409 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:40.232652 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b96e1d-ae2d-442d-9fdc-b72172bf4409 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:40.244076 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b96e1d-ae2d-442d-9fdc-b72172bf4409 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:40.245559 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b96e1d-ae2d-442d-9fdc-b72172bf4409 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:40.252642 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220bcb1c-bd9d-4fe8-bf27-68eea6271642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:40.254838 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220bcb1c-bd9d-4fe8-bf27-68eea6271642 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:40.255122 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220bcb1c-bd9d-4fe8-bf27-68eea6271642 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:40.255398 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220bcb1c-bd9d-4fe8-bf27-68eea6271642 None admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:40.256094 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220bcb1c-bd9d-4fe8-bf27-68eea6271642 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:40.256406 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220bcb1c-bd9d-4fe8-bf27-68eea6271642 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:40.257870 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220bcb1c-bd9d-4fe8-bf27-68eea6271642 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:40.266551 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:40.267259 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-55b96e1d-ae2d-442d-9fdc-b72172bf4409 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:40.267259 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:40.267259 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:40.267259 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:40.267259 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:40.267259 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:40.267259 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:40.267259 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:40.267259 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:40.267259 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:40.267259 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:40.267259 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:40.267259 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:18:40.267259 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:18:40.267259 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:18:40.267259 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:18:40.268163 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:40.268163 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:40.268163 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:40.268163 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:40.268163 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:40.268163 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:40.268163 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:40.268163 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:40.268163 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:40.268163 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:40.268163 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:40.268163 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:40.268163 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:40.268163 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:40.268163 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:40.268163 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:40.268163 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:18:40.269081 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:18:40.269081 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:40.269081 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:18:40.272307 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 373/760] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:18:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:18:40.273896 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b857fd4e-8a20-4f2f-8179-ec0c19400a6a None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:40.274316 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b857fd4e-8a20-4f2f-8179-ec0c19400a6a None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:40.274716 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b857fd4e-8a20-4f2f-8179-ec0c19400a6a None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:40.280226 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 374/761] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:18:40 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:18:40.280226 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b857fd4e-8a20-4f2f-8179-ec0c19400a6a None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:40.283076 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 388/762] 10.222.0.102 () {56 vars in 1153 bytes} [Thu Sep 1 17:18:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:40.284216 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9fb050-b448-4318-92fd-a26c6b2bf79a None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:40.285035 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9fb050-b448-4318-92fd-a26c6b2bf79a None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:40.285353 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9fb050-b448-4318-92fd-a26c6b2bf79a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:40.303566 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-3b9fb050-b448-4318-92fd-a26c6b2bf79a None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:18:40.304760 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-3b9fb050-b448-4318-92fd-a26c6b2bf79a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:18:40.325268 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:40.377370 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 389/763] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:18:40 2022] POST /identity/v3/auth/tokens => generated 1003 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:18:40.377370 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9fb050-b448-4318-92fd-a26c6b2bf79a None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:40.441839 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b857fd4e-8a20-4f2f-8179-ec0c19400a6a None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:40.510638 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16187ac5-e33a-460c-8a15-324f047cf1d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:40.511554 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16187ac5-e33a-460c-8a15-324f047cf1d1 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:40.511986 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16187ac5-e33a-460c-8a15-324f047cf1d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:40.512422 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16187ac5-e33a-460c-8a15-324f047cf1d1 None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:40.513226 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16187ac5-e33a-460c-8a15-324f047cf1d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:40.513715 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16187ac5-e33a-460c-8a15-324f047cf1d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:40.516932 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16187ac5-e33a-460c-8a15-324f047cf1d1 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:40.541252 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 375/764] 10.222.0.102 () {56 vars in 1114 bytes} [Thu Sep 1 17:18:40 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:40.548917 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16187ac5-e33a-460c-8a15-324f047cf1d1 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:40.568440 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9fb050-b448-4318-92fd-a26c6b2bf79a None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:40.613974 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:40.631269 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7a5801-2fb1-4379-8b51-b81bf1b07c2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:40.632113 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7a5801-2fb1-4379-8b51-b81bf1b07c2b None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:40.632541 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7a5801-2fb1-4379-8b51-b81bf1b07c2b None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:40.634583 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7a5801-2fb1-4379-8b51-b81bf1b07c2b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:40.635938 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7a5801-2fb1-4379-8b51-b81bf1b07c2b None admin] RBAC: Adding query filter params (role.id=f21e23654e4740158c7d4f46e726e515, scope.project.id=e78fff7f1f6e4f68b8d233ad2e5bbd4f, user.id=575a76b99ea945efb1024fbb3dc81aaf) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:40.637263 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7a5801-2fb1-4379-8b51-b81bf1b07c2b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:40.638918 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7a5801-2fb1-4379-8b51-b81bf1b07c2b None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:40.651433 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 390/765] 10.222.0.102 () {56 vars in 1379 bytes} [Thu Sep 1 17:18:40 2022] GET /identity/v3/role_assignments?user.id=575a76b99ea945efb1024fbb3dc81aaf&scope.project.id=e78fff7f1f6e4f68b8d233ad2e5bbd4f&role.id=f21e23654e4740158c7d4f46e726e515 => generated 260 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:40.699457 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:40.707348 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16187ac5-e33a-460c-8a15-324f047cf1d1 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:40.730764 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7a5801-2fb1-4379-8b51-b81bf1b07c2b None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:40.763793 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eeff7d-c7b3-4bba-9edd-607826a611a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:40.764362 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eeff7d-c7b3-4bba-9edd-607826a611a9 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:40.764643 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eeff7d-c7b3-4bba-9edd-607826a611a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:40.764913 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eeff7d-c7b3-4bba-9edd-607826a611a9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:40.779406 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5eeff7d-c7b3-4bba-9edd-607826a611a9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:40.781792 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5eeff7d-c7b3-4bba-9edd-607826a611a9 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:40.805211 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-a5eeff7d-c7b3-4bba-9edd-607826a611a9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:18:40.805211 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:40.805211 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:40.805211 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:40.805211 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:40.805211 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:40.805211 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:40.805211 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:40.805211 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:40.805211 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:40.805211 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:40.805211 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:40.805211 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:18:40.805211 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:18:40.805211 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:18:40.806193 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:18:40.806193 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:40.806193 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:40.806193 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:40.806193 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:40.806193 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:40.806193 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:40.806193 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:40.806193 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:40.806193 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:40.806193 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:40.806193 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:40.806193 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:40.806193 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:40.806193 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:40.806193 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:40.806193 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:40.807118 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:18:40.807118 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:18:40.807118 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:18:40.807118 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:18:40.807118 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:18:40.807118 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:18:40.807118 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:18:40.807118 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:18:40.813608 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 376/766] 10.222.0.102 () {56 vars in 1116 bytes} [Thu Sep 1 17:18:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:40.814216 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eeff7d-c7b3-4bba-9edd-607826a611a9 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:40.916641 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7a5801-2fb1-4379-8b51-b81bf1b07c2b None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:40.979270 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eeff7d-c7b3-4bba-9edd-607826a611a9 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:40.995542 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43d69fb-01f5-4eb7-adb9-c930b04f28ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:40.996211 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43d69fb-01f5-4eb7-adb9-c930b04f28ee None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:40.996444 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43d69fb-01f5-4eb7-adb9-c930b04f28ee None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:40.996772 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43d69fb-01f5-4eb7-adb9-c930b04f28ee None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:40.997292 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43d69fb-01f5-4eb7-adb9-c930b04f28ee None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:40.998791 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43d69fb-01f5-4eb7-adb9-c930b04f28ee None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:41.011123 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-a43d69fb-01f5-4eb7-adb9-c930b04f28ee None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 17:18:41.011123 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:41.011123 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 17:18:41.011123 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 17:18:41.011123 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 17:18:41.011123 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 17:18:41.011123 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 17:18:41.011123 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:18:41.011123 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 17:18:41.011123 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:18:41.011123 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 17:18:41.011123 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:18:41.011123 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:41.011123 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:41.011123 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:41.011123 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:41.011123 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:41.012114 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:41.012114 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:41.012114 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:41.012114 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:41.012114 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:41.012114 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:41.012114 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 17:18:41.012114 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 17:18:41.012114 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 17:18:41.012114 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 17:18:41.012114 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:41.012114 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:41.012114 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:41.012114 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:41.012114 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:41.012114 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:41.012938 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:41.012938 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:41.012938 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:41.012938 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:41.012938 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:41.012938 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:41.012938 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:41.012938 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:41.012938 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:41.012938 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:41.012938 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 17:18:41.012938 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:18:41.012938 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 17:18:41.012938 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:18:41.012938 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 391/767] 10.222.0.102 () {56 vars in 1134 bytes} [Thu Sep 1 17:18:40 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:18:41.019416 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6690ac52-6b1f-4afa-ac2a-4e788d774ef2 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:41.019416 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6690ac52-6b1f-4afa-ac2a-4e788d774ef2 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:41.019416 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6690ac52-6b1f-4afa-ac2a-4e788d774ef2 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:41.020167 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 392/768] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:18:41 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:18:41.021037 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6690ac52-6b1f-4afa-ac2a-4e788d774ef2 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:41.027631 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f9968b-a7a7-4a85-994d-0eded5aba8df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:41.028558 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f9968b-a7a7-4a85-994d-0eded5aba8df None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:41.029037 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f9968b-a7a7-4a85-994d-0eded5aba8df None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:41.029574 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f9968b-a7a7-4a85-994d-0eded5aba8df None admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:41.030465 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f9968b-a7a7-4a85-994d-0eded5aba8df None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:41.030970 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f9968b-a7a7-4a85-994d-0eded5aba8df None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:41.032626 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f9968b-a7a7-4a85-994d-0eded5aba8df None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:41.045180 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 377/769] 10.222.0.102 () {56 vars in 1153 bytes} [Thu Sep 1 17:18:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:41.046656 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6bb12f-87d5-46e2-88a3-53de81a15c67 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:41.047124 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6bb12f-87d5-46e2-88a3-53de81a15c67 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:41.047579 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6bb12f-87d5-46e2-88a3-53de81a15c67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:41.066318 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-3a6bb12f-87d5-46e2-88a3-53de81a15c67 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:18:41.067724 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-3a6bb12f-87d5-46e2-88a3-53de81a15c67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:18:41.079342 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:41.082208 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:41.131858 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6690ac52-6b1f-4afa-ac2a-4e788d774ef2 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:41.145561 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 378/770] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:18:41 2022] POST /identity/v3/auth/tokens => generated 1003 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:18:41.146456 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6bb12f-87d5-46e2-88a3-53de81a15c67 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:41.167387 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e08948b-510a-4e55-a90b-48c4c1e3ea6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:41.167990 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e08948b-510a-4e55-a90b-48c4c1e3ea6b None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:41.168268 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e08948b-510a-4e55-a90b-48c4c1e3ea6b None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:41.168539 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e08948b-510a-4e55-a90b-48c4c1e3ea6b None admin] PATH_INFO: `/v3/services` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:41.169166 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e08948b-510a-4e55-a90b-48c4c1e3ea6b None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:41.169552 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e08948b-510a-4e55-a90b-48c4c1e3ea6b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:41.170971 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e08948b-510a-4e55-a90b-48c4c1e3ea6b None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:41.176714 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 393/771] 10.222.0.102 () {56 vars in 1129 bytes} [Thu Sep 1 17:18:41 2022] GET /identity/v3/services?name=block-storage => generated 131 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:41.186524 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e08948b-510a-4e55-a90b-48c4c1e3ea6b None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:41.326884 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6bb12f-87d5-46e2-88a3-53de81a15c67 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:41.373070 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e08948b-510a-4e55-a90b-48c4c1e3ea6b None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:41.396085 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f2139a-99b0-40b5-b3dc-2385e31104aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:41.396757 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f2139a-99b0-40b5-b3dc-2385e31104aa None admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:41.397038 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f2139a-99b0-40b5-b3dc-2385e31104aa None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:41.397340 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f2139a-99b0-40b5-b3dc-2385e31104aa None admin] PATH_INFO: `/v3/projects/e78fff7f1f6e4f68b8d233ad2e5bbd4f/users/5f8293316baf4774b62fc11d55d50865/roles/36577a6f46d642d4b81805ede9872490` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:41.401773 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f2139a-99b0-40b5-b3dc-2385e31104aa None admin] RBAC: Authorizing `identity:create_grant(project_id=e78fff7f1f6e4f68b8d233ad2e5bbd4f, user_id=5f8293316baf4774b62fc11d55d50865, role_id=36577a6f46d642d4b81805ede9872490)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:41.403207 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f2139a-99b0-40b5-b3dc-2385e31104aa None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:41.423810 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 379/772] 10.222.0.102 () {58 vars in 1444 bytes} [Thu Sep 1 17:18:41 2022] PUT /identity/v3/projects/e78fff7f1f6e4f68b8d233ad2e5bbd4f/users/5f8293316baf4774b62fc11d55d50865/roles/36577a6f46d642d4b81805ede9872490 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:18:41.426188 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:41.439919 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4383b6f9-e21e-4850-bede-19023cf0e2bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:41.440774 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4383b6f9-e21e-4850-bede-19023cf0e2bb None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:41.441681 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4383b6f9-e21e-4850-bede-19023cf0e2bb None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:41.442046 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4383b6f9-e21e-4850-bede-19023cf0e2bb None admin] PATH_INFO: `/v3/services` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:41.442787 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4383b6f9-e21e-4850-bede-19023cf0e2bb None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:41.443282 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4383b6f9-e21e-4850-bede-19023cf0e2bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:41.444852 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4383b6f9-e21e-4850-bede-19023cf0e2bb None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:41.452050 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 394/773] 10.222.0.102 () {56 vars in 1129 bytes} [Thu Sep 1 17:18:41 2022] GET /identity/v3/services?type=block-storage => generated 369 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:41.457727 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f2139a-99b0-40b5-b3dc-2385e31104aa None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:41.467613 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4383b6f9-e21e-4850-bede-19023cf0e2bb None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:41.476586 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:41.626288 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4383b6f9-e21e-4850-bede-19023cf0e2bb None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:41.651770 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f2139a-99b0-40b5-b3dc-2385e31104aa None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:41.670204 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dba64ce-324a-4275-835e-a6b148f6df55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:41.670847 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dba64ce-324a-4275-835e-a6b148f6df55 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:41.671127 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dba64ce-324a-4275-835e-a6b148f6df55 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:41.671400 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dba64ce-324a-4275-835e-a6b148f6df55 None admin] PATH_INFO: `/v3/services/network` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:41.671988 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dba64ce-324a-4275-835e-a6b148f6df55 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:41.673587 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dba64ce-324a-4275-835e-a6b148f6df55 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:41.684465 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-5dba64ce-324a-4275-835e-a6b148f6df55 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 17:18:41.684465 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:41.684465 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 17:18:41.684465 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 17:18:41.684465 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 17:18:41.684465 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 17:18:41.684465 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 17:18:41.684465 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:18:41.684465 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 17:18:41.684465 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:18:41.684465 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 17:18:41.684465 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:18:41.684465 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:41.684465 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:41.684465 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:41.684465 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:41.684465 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:41.685518 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:41.685518 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:41.685518 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:41.685518 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:41.685518 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:41.685518 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:41.685518 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 17:18:41.685518 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 17:18:41.685518 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 17:18:41.685518 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 17:18:41.685518 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:41.685518 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:41.685518 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:41.685518 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:41.685518 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:41.685518 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:41.686354 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:41.686354 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:41.686354 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:41.686354 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:41.686354 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:41.686354 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:41.686354 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:41.686354 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:41.686354 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:41.686354 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:41.686354 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 17:18:41.686354 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:18:41.686354 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 17:18:41.686354 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:18:41.686354 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 395/774] 10.222.0.102 () {56 vars in 1116 bytes} [Thu Sep 1 17:18:41 2022] GET /identity/v3/services/network => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:18:41.692254 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dba64ce-324a-4275-835e-a6b148f6df55 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:41.692689 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732525ab-e21f-4851-a008-854e4948ee14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:41.693330 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732525ab-e21f-4851-a008-854e4948ee14 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:41.693624 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732525ab-e21f-4851-a008-854e4948ee14 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:41.693901 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732525ab-e21f-4851-a008-854e4948ee14 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:41.694612 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732525ab-e21f-4851-a008-854e4948ee14 None admin] RBAC: Adding query filter params (interface=public, service_id=c71880e5e00748c0832122e72c285502, region_id=RegionOne) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:41.694924 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732525ab-e21f-4851-a008-854e4948ee14 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:41.696573 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732525ab-e21f-4851-a008-854e4948ee14 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:41.705365 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 380/775] 10.222.0.102 () {56 vars in 1256 bytes} [Thu Sep 1 17:18:41 2022] GET /identity/v3/endpoints?service_id=c71880e5e00748c0832122e72c285502&interface=public®ion_id=RegionOne => generated 195 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:41.710449 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732525ab-e21f-4851-a008-854e4948ee14 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:41.752101 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:41.793691 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:41.823101 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732525ab-e21f-4851-a008-854e4948ee14 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:41.834489 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dba64ce-324a-4275-835e-a6b148f6df55 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:41.876529 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4c5ff4-8e12-475f-a7a7-191ec0ab9c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:41.877175 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4c5ff4-8e12-475f-a7a7-191ec0ab9c8a None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:41.877590 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4c5ff4-8e12-475f-a7a7-191ec0ab9c8a None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:41.878216 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4c5ff4-8e12-475f-a7a7-191ec0ab9c8a None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:41.878896 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4c5ff4-8e12-475f-a7a7-191ec0ab9c8a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:41.880894 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4c5ff4-8e12-475f-a7a7-191ec0ab9c8a None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:41.886296 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdb6865-0e1f-4454-b5c7-248b054ae7e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:41.886994 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 381/776] 10.222.0.102 () {56 vars in 1092 bytes} [Thu Sep 1 17:18:41 2022] GET /identity/v3/services => generated 776 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:18:41.887445 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdb6865-0e1f-4454-b5c7-248b054ae7e6 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:41.887984 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdb6865-0e1f-4454-b5c7-248b054ae7e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:41.888261 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579016c4-1782-450e-a541-3fb7181ac101 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:41.888550 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579016c4-1782-450e-a541-3fb7181ac101 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:41.888828 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579016c4-1782-450e-a541-3fb7181ac101 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:41.889082 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdb6865-0e1f-4454-b5c7-248b054ae7e6 None admin] PATH_INFO: `/v3/services` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:41.889955 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdb6865-0e1f-4454-b5c7-248b054ae7e6 None admin] RBAC: Adding query filter params (name=network) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:41.894274 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 382/777] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:18:41 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:18:41.894485 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdb6865-0e1f-4454-b5c7-248b054ae7e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:41.895306 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdb6865-0e1f-4454-b5c7-248b054ae7e6 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:41.898169 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faad5d2a-6175-4fd9-84f5-0533661286bd None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:41.898442 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faad5d2a-6175-4fd9-84f5-0533661286bd None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:41.899133 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faad5d2a-6175-4fd9-84f5-0533661286bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:41.902809 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 396/778] 10.222.0.102 () {56 vars in 1117 bytes} [Thu Sep 1 17:18:41 2022] GET /identity/v3/services?name=network => generated 125 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:41.908464 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdb6865-0e1f-4454-b5c7-248b054ae7e6 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:41.933942 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-faad5d2a-6175-4fd9-84f5-0533661286bd None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:18:41.935193 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-faad5d2a-6175-4fd9-84f5-0533661286bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:18:42.079456 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdb6865-0e1f-4454-b5c7-248b054ae7e6 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:42.120715 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c052aa86-22a4-48ac-b6fc-ec546aa2f44e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:42.121376 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c052aa86-22a4-48ac-b6fc-ec546aa2f44e None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:42.121663 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c052aa86-22a4-48ac-b6fc-ec546aa2f44e None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:42.121938 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c052aa86-22a4-48ac-b6fc-ec546aa2f44e None admin] PATH_INFO: `/v3/services` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:42.122584 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c052aa86-22a4-48ac-b6fc-ec546aa2f44e None admin] RBAC: Adding query filter params (type=network) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:42.122892 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c052aa86-22a4-48ac-b6fc-ec546aa2f44e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:42.124294 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c052aa86-22a4-48ac-b6fc-ec546aa2f44e None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:42.131060 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 397/779] 10.222.0.102 () {56 vars in 1117 bytes} [Thu Sep 1 17:18:41 2022] GET /identity/v3/services?type=network => generated 352 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:42.133714 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:42.136623 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c052aa86-22a4-48ac-b6fc-ec546aa2f44e None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:42.184875 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 383/780] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:18:41 2022] POST /identity/v3/auth/tokens => generated 1003 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:18:42.271302 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c052aa86-22a4-48ac-b6fc-ec546aa2f44e None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:42.322047 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7fdf6a-5ea0-461d-8c7b-0775be8f81e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:42.323037 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7fdf6a-5ea0-461d-8c7b-0775be8f81e7 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:42.323522 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7fdf6a-5ea0-461d-8c7b-0775be8f81e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:42.323995 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7fdf6a-5ea0-461d-8c7b-0775be8f81e7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:42.324915 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7fdf6a-5ea0-461d-8c7b-0775be8f81e7 None admin] RBAC: Adding query filter params (interface=public, service_id=eb3067963f0241eea9a78672daebf522, region_id=RegionOne) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:42.325478 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7fdf6a-5ea0-461d-8c7b-0775be8f81e7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:42.327091 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7fdf6a-5ea0-461d-8c7b-0775be8f81e7 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:42.340977 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 398/781] 10.222.0.102 () {56 vars in 1256 bytes} [Thu Sep 1 17:18:42 2022] GET /identity/v3/endpoints?service_id=eb3067963f0241eea9a78672daebf522&interface=public®ion_id=RegionOne => generated 195 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:42.346680 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faad5d2a-6175-4fd9-84f5-0533661286bd None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:42.430021 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:42.478516 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faad5d2a-6175-4fd9-84f5-0533661286bd None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:42.528731 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f3f759-29d3-4b5c-a6ae-c76152f943b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:42.529431 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f3f759-29d3-4b5c-a6ae-c76152f943b2 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:42.529715 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f3f759-29d3-4b5c-a6ae-c76152f943b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:42.529988 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f3f759-29d3-4b5c-a6ae-c76152f943b2 None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:42.530597 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f3f759-29d3-4b5c-a6ae-c76152f943b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:42.531965 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f3f759-29d3-4b5c-a6ae-c76152f943b2 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:42.542247 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 384/782] 10.222.0.102 () {56 vars in 1092 bytes} [Thu Sep 1 17:18:42 2022] GET /identity/v3/services => generated 776 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:42.577456 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7fdf6a-5ea0-461d-8c7b-0775be8f81e7 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:42.625943 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:42.696912 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7fdf6a-5ea0-461d-8c7b-0775be8f81e7 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:42.740913 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4add50c1-8360-4161-ab5f-68ff3225abc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:42.741849 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4add50c1-8360-4161-ab5f-68ff3225abc2 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:42.742296 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4add50c1-8360-4161-ab5f-68ff3225abc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:42.742713 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4add50c1-8360-4161-ab5f-68ff3225abc2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:42.754907 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4add50c1-8360-4161-ab5f-68ff3225abc2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:18:42.760094 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-4add50c1-8360-4161-ab5f-68ff3225abc2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:18:42.760094 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:42.760094 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:42.760094 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:42.760094 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:42.760094 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:42.760094 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:42.760094 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:42.760094 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:42.760094 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:42.760094 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:42.760094 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:42.760094 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:18:42.760094 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:18:42.760094 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:18:42.760094 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:18:42.761201 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:18:42.761201 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:18:42.761201 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:18:42.761201 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:18:42.761201 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:42.761201 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:42.761201 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:42.761201 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:42.761201 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:42.761201 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:42.761201 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:42.761201 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:42.761201 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:42.761201 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:42.761201 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:42.761201 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:42.762248 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:42.762248 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:42.762248 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:42.762248 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:42.762248 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:18:42.762248 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:18:42.762248 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:18:42.762248 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:18:42.762248 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:18:42.762248 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:18:42.762248 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:18:42.762248 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:18:42.764403 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 399/783] 10.222.0.102 () {56 vars in 1101 bytes} [Thu Sep 1 17:18:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:42.774408 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f3f759-29d3-4b5c-a6ae-c76152f943b2 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:42.886717 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f3f759-29d3-4b5c-a6ae-c76152f943b2 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:42.914921 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:42.927732 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca106c9-d1ed-4b56-ae46-546b55d5fbca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:42.928347 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca106c9-d1ed-4b56-ae46-546b55d5fbca None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:42.928623 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca106c9-d1ed-4b56-ae46-546b55d5fbca None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:42.928895 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca106c9-d1ed-4b56-ae46-546b55d5fbca None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:42.929579 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca106c9-d1ed-4b56-ae46-546b55d5fbca None admin] RBAC: Adding query filter params (name=admin) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:42.929883 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca106c9-d1ed-4b56-ae46-546b55d5fbca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:42.931249 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca106c9-d1ed-4b56-ae46-546b55d5fbca None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:42.940310 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 385/784] 10.222.0.102 () {56 vars in 1104 bytes} [Thu Sep 1 17:18:42 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:42.946013 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4add50c1-8360-4161-ab5f-68ff3225abc2 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:43.008232 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be12ed36-08e0-48f3-902a-e557c00eaba0 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:43.008513 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be12ed36-08e0-48f3-902a-e557c00eaba0 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:43.008709 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be12ed36-08e0-48f3-902a-e557c00eaba0 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:43.010075 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 386/785] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:18:43 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:18:43.031128 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c31e14-94af-4569-8696-03e4ccec9286 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:43.031444 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c31e14-94af-4569-8696-03e4ccec9286 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:43.031679 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c31e14-94af-4569-8696-03e4ccec9286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:43.061398 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-b4c31e14-94af-4569-8696-03e4ccec9286 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:18:43.062825 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-b4c31e14-94af-4569-8696-03e4ccec9286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:18:43.065688 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:43.068811 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4add50c1-8360-4161-ab5f-68ff3225abc2 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:43.105252 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616fde0e-a68a-479b-b97c-fcab5d29fe04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:43.105929 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616fde0e-a68a-479b-b97c-fcab5d29fe04 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:43.106157 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616fde0e-a68a-479b-b97c-fcab5d29fe04 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:43.106666 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616fde0e-a68a-479b-b97c-fcab5d29fe04 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:43.118117 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616fde0e-a68a-479b-b97c-fcab5d29fe04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:43.119740 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616fde0e-a68a-479b-b97c-fcab5d29fe04 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:43.135271 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-616fde0e-a68a-479b-b97c-fcab5d29fe04 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:43.135271 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:43.135271 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:43.135271 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:43.135271 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:43.135271 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:43.135271 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:43.135271 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:43.135271 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:43.135271 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:43.135271 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:43.135271 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:43.135271 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:18:43.135271 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:18:43.135271 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:18:43.135271 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:18:43.136259 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:43.136259 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:43.136259 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:43.136259 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:43.136259 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:43.136259 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:43.136259 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:43.136259 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:43.136259 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:43.136259 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:43.136259 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:43.136259 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:43.136259 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:43.136259 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:43.136259 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:43.136259 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:43.136259 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:18:43.137132 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:18:43.137132 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:43.137132 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:18:43.137132 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 400/786] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:18:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:43.162876 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616fde0e-a68a-479b-b97c-fcab5d29fe04 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:43.179055 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 387/787] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:18:43 2022] POST /identity/v3/auth/tokens => generated 1003 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 17:18:43.310046 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616fde0e-a68a-479b-b97c-fcab5d29fe04 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:43.354394 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec2e93c-476e-4a8d-a05e-5921c57e9b9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:43.355395 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec2e93c-476e-4a8d-a05e-5921c57e9b9b None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:43.355881 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec2e93c-476e-4a8d-a05e-5921c57e9b9b None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:43.356353 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec2e93c-476e-4a8d-a05e-5921c57e9b9b None admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:43.357194 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec2e93c-476e-4a8d-a05e-5921c57e9b9b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:43.357733 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec2e93c-476e-4a8d-a05e-5921c57e9b9b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:43.359310 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec2e93c-476e-4a8d-a05e-5921c57e9b9b None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:43.376264 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 401/788] 10.222.0.102 () {56 vars in 1114 bytes} [Thu Sep 1 17:18:43 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:43.381960 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c31e14-94af-4569-8696-03e4ccec9286 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:43.485719 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec2e93c-476e-4a8d-a05e-5921c57e9b9b None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:43.501494 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:43.510751 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c31e14-94af-4569-8696-03e4ccec9286 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:43.546543 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:43.573964 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d78f908-d108-4bf4-be9b-ed8f9f22ff25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:43.574516 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d78f908-d108-4bf4-be9b-ed8f9f22ff25 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:43.574791 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d78f908-d108-4bf4-be9b-ed8f9f22ff25 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:43.575057 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d78f908-d108-4bf4-be9b-ed8f9f22ff25 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:43.586457 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d78f908-d108-4bf4-be9b-ed8f9f22ff25 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:43.587894 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d78f908-d108-4bf4-be9b-ed8f9f22ff25 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:43.610196 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-3d78f908-d108-4bf4-be9b-ed8f9f22ff25 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 01 17:18:43.610196 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:43.610196 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:43.610196 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:43.610196 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:43.610196 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:43.610196 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:43.610196 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:43.610196 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:43.610196 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:43.610196 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:43.610196 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:43.610196 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:18:43.610196 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:18:43.610196 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:18:43.610196 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:18:43.611216 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:43.611216 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:43.611216 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:18:43.611216 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:18:43.611216 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:18:43.611216 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:18:43.611216 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:43.611216 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:43.611216 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:43.611216 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:43.611216 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:43.611216 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:43.611216 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:43.611216 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:43.611216 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:43.611216 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:43.611216 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:43.611216 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:43.612128 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:43.612128 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:43.612128 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:18:43.612128 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:18:43.612128 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:18:43.612128 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:18:43.612128 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:18:43.612128 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:18:43.612128 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 01 17:18:43.612128 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:18:43.612128 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 388/789] 10.222.0.102 () {56 vars in 1101 bytes} [Thu Sep 1 17:18:43 2022] GET /identity/v3/users/swift => generated 83 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:43.621034 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d78f908-d108-4bf4-be9b-ed8f9f22ff25 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:43.667703 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec2e93c-476e-4a8d-a05e-5921c57e9b9b None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:43.730440 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe63543-beea-4b99-8394-9419b9d09f7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:43.731077 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe63543-beea-4b99-8394-9419b9d09f7e None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:43.731348 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe63543-beea-4b99-8394-9419b9d09f7e None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:43.731618 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe63543-beea-4b99-8394-9419b9d09f7e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:43.753860 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe63543-beea-4b99-8394-9419b9d09f7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:18:43.759877 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-efe63543-beea-4b99-8394-9419b9d09f7e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:18:43.759877 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:43.759877 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:43.759877 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:43.759877 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:43.759877 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:43.759877 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:43.759877 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:43.759877 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:43.759877 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:43.759877 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:43.759877 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:43.759877 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:18:43.759877 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:18:43.759877 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:18:43.759877 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:18:43.760878 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:18:43.760878 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:18:43.760878 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:18:43.760878 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:18:43.760878 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:43.760878 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:43.760878 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:43.760878 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:43.760878 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:43.760878 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:43.760878 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:43.760878 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:43.760878 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:43.760878 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:43.760878 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:43.760878 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:43.761737 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:43.761737 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:43.761737 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:43.761737 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:43.761737 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:18:43.761737 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:18:43.761737 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:18:43.761737 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:18:43.761737 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:18:43.761737 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:18:43.761737 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:18:43.761737 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:18:43.766656 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 402/790] 10.222.0.102 () {56 vars in 1107 bytes} [Thu Sep 1 17:18:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:43.769977 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe63543-beea-4b99-8394-9419b9d09f7e None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:43.802750 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d78f908-d108-4bf4-be9b-ed8f9f22ff25 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:43.874485 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:43.909797 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe63543-beea-4b99-8394-9419b9d09f7e None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:43.926242 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ce7ce5-7e29-4bd2-8ad1-cae83957bf92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:43.926864 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ce7ce5-7e29-4bd2-8ad1-cae83957bf92 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:43.927143 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ce7ce5-7e29-4bd2-8ad1-cae83957bf92 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:43.927419 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ce7ce5-7e29-4bd2-8ad1-cae83957bf92 None admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:43.928144 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ce7ce5-7e29-4bd2-8ad1-cae83957bf92 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:43.928505 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ce7ce5-7e29-4bd2-8ad1-cae83957bf92 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:43.929964 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ce7ce5-7e29-4bd2-8ad1-cae83957bf92 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:43.954765 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a698ba10-f009-4d12-993e-421ba90316a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:43.955379 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a698ba10-f009-4d12-993e-421ba90316a6 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:43.955660 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a698ba10-f009-4d12-993e-421ba90316a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:43.955933 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a698ba10-f009-4d12-993e-421ba90316a6 None admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:43.957165 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a698ba10-f009-4d12-993e-421ba90316a6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:43.957495 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a698ba10-f009-4d12-993e-421ba90316a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:43.958964 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a698ba10-f009-4d12-993e-421ba90316a6 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:43.965023 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 389/791] 10.222.0.102 () {56 vars in 1140 bytes} [Thu Sep 1 17:18:43 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 372 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:43.966558 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8628f2-5097-493e-92c4-35b6b7f66eb5 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:43.967154 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8628f2-5097-493e-92c4-35b6b7f66eb5 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:43.967691 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8628f2-5097-493e-92c4-35b6b7f66eb5 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:43.973347 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 390/792] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:18:43 2022] GET /identity => generated 271 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:18:43.973719 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 403/793] 10.222.0.102 () {56 vars in 1108 bytes} [Thu Sep 1 17:18:43 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:43.974607 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8628f2-5097-493e-92c4-35b6b7f66eb5 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:43.974963 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56352626-c9a9-420a-b235-27b29f7e3955 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:43.975249 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56352626-c9a9-420a-b235-27b29f7e3955 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:43.975523 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56352626-c9a9-420a-b235-27b29f7e3955 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:43.997668 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-56352626-c9a9-420a-b235-27b29f7e3955 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:18:43.999333 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-56352626-c9a9-420a-b235-27b29f7e3955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:18:44.024623 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:44.072932 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 404/794] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:18:43 2022] POST /identity/v3/auth/tokens => generated 1003 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:18:44.073896 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56352626-c9a9-420a-b235-27b29f7e3955 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:44.092518 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8628f2-5097-493e-92c4-35b6b7f66eb5 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:44.129803 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0632d5a-b9ce-43a5-af86-49020e32da25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:44.130878 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0632d5a-b9ce-43a5-af86-49020e32da25 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:44.131362 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0632d5a-b9ce-43a5-af86-49020e32da25 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:44.131898 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0632d5a-b9ce-43a5-af86-49020e32da25 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:44.138682 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:44.144383 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0632d5a-b9ce-43a5-af86-49020e32da25 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:44.146135 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0632d5a-b9ce-43a5-af86-49020e32da25 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:44.162490 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-d0632d5a-b9ce-43a5-af86-49020e32da25 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:44.162490 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:44.162490 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:44.162490 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:44.162490 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:44.162490 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:44.162490 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:44.162490 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:44.162490 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:44.162490 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:44.162490 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:44.162490 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:44.162490 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:18:44.162490 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:18:44.162490 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:18:44.162490 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:18:44.163455 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:44.163455 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:44.163455 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:44.163455 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:44.163455 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:44.163455 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:44.163455 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:44.163455 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:44.163455 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:44.163455 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:44.163455 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:44.163455 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:44.163455 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:44.163455 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:44.163455 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:44.163455 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:44.163455 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:18:44.164313 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:18:44.164313 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:44.164313 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:18:44.166536 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 391/795] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:18:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:44.173131 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0632d5a-b9ce-43a5-af86-49020e32da25 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:44.187285 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56352626-c9a9-420a-b235-27b29f7e3955 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:44.225206 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a62e561-3cee-4bbf-ad7e-5ed4db8d092b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:44.225860 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a62e561-3cee-4bbf-ad7e-5ed4db8d092b None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:44.226141 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a62e561-3cee-4bbf-ad7e-5ed4db8d092b None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:44.226417 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a62e561-3cee-4bbf-ad7e-5ed4db8d092b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:44.237344 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a62e561-3cee-4bbf-ad7e-5ed4db8d092b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:44.238810 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a62e561-3cee-4bbf-ad7e-5ed4db8d092b None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:44.254922 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-9a62e561-3cee-4bbf-ad7e-5ed4db8d092b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:44.254922 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:44.254922 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:44.254922 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:44.254922 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:44.254922 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:44.254922 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:44.254922 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:44.254922 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:44.254922 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:44.254922 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:44.254922 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:44.254922 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:18:44.254922 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:18:44.254922 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:18:44.254922 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:18:44.255904 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:44.255904 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:44.255904 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:44.255904 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:44.255904 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:44.255904 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:44.255904 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:44.255904 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:44.255904 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:44.255904 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:44.255904 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:44.255904 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:44.255904 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:44.255904 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:44.255904 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:44.255904 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:44.255904 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:18:44.257195 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:18:44.257195 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:44.257195 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:18:44.257195 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 405/796] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:18:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:44.263900 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a62e561-3cee-4bbf-ad7e-5ed4db8d092b None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:44.369633 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a62e561-3cee-4bbf-ad7e-5ed4db8d092b None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:44.372141 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:44.391447 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:44.400099 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0632d5a-b9ce-43a5-af86-49020e32da25 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:44.414561 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87230b4-e6a5-4b38-aeed-8a674e1235bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:44.415173 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87230b4-e6a5-4b38-aeed-8a674e1235bd None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:44.415501 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87230b4-e6a5-4b38-aeed-8a674e1235bd None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:44.415722 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87230b4-e6a5-4b38-aeed-8a674e1235bd None admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:44.416366 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87230b4-e6a5-4b38-aeed-8a674e1235bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:44.416671 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87230b4-e6a5-4b38-aeed-8a674e1235bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:44.418082 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87230b4-e6a5-4b38-aeed-8a674e1235bd None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:44.437345 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 406/797] 10.222.0.102 () {56 vars in 1114 bytes} [Thu Sep 1 17:18:44 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:44.437345 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617ac1ba-d6f7-4742-a3e6-cf1d103cc12b None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:44.437345 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617ac1ba-d6f7-4742-a3e6-cf1d103cc12b None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:44.437345 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617ac1ba-d6f7-4742-a3e6-cf1d103cc12b None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:44.437345 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 407/798] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:18:44 2022] GET /identity => generated 271 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:18:44.437345 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617ac1ba-d6f7-4742-a3e6-cf1d103cc12b None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:44.449742 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f42ec05-103b-42e7-8b1c-23ce910ee7aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:44.450613 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f42ec05-103b-42e7-8b1c-23ce910ee7aa None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:44.451041 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f42ec05-103b-42e7-8b1c-23ce910ee7aa None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:44.451458 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f42ec05-103b-42e7-8b1c-23ce910ee7aa None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:44.452348 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f42ec05-103b-42e7-8b1c-23ce910ee7aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:44.452891 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f42ec05-103b-42e7-8b1c-23ce910ee7aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:44.454435 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f42ec05-103b-42e7-8b1c-23ce910ee7aa None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:44.468375 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 392/799] 10.222.0.102 () {56 vars in 1114 bytes} [Thu Sep 1 17:18:44 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:44.470198 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b256af79-610d-4978-b159-85170eb68ce8 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:44.470665 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b256af79-610d-4978-b159-85170eb68ce8 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:44.470940 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b256af79-610d-4978-b159-85170eb68ce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:44.489570 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-b256af79-610d-4978-b159-85170eb68ce8 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:18:44.490771 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-b256af79-610d-4978-b159-85170eb68ce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:18:44.551618 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617ac1ba-d6f7-4742-a3e6-cf1d103cc12b None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:44.569420 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 393/800] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:18:44 2022] POST /identity/v3/auth/tokens => generated 1003 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:18:44.569420 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b256af79-610d-4978-b159-85170eb68ce8 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:44.591841 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82683d50-ba3a-4db2-b5bd-13b326d5a836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:44.592472 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82683d50-ba3a-4db2-b5bd-13b326d5a836 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:44.592753 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82683d50-ba3a-4db2-b5bd-13b326d5a836 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:44.593026 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82683d50-ba3a-4db2-b5bd-13b326d5a836 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:44.600918 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82683d50-ba3a-4db2-b5bd-13b326d5a836 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:44.602382 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82683d50-ba3a-4db2-b5bd-13b326d5a836 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:44.616591 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-82683d50-ba3a-4db2-b5bd-13b326d5a836 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 17:18:44.616591 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:44.616591 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:44.616591 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:44.616591 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:44.616591 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:44.616591 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:44.616591 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:44.616591 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:44.616591 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:44.616591 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:44.616591 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:44.616591 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:18:44.616591 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:18:44.616591 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:18:44.616591 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:18:44.617845 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:44.617845 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:44.617845 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:18:44.617845 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:18:44.617845 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:18:44.617845 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:18:44.617845 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:44.617845 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:44.617845 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:44.617845 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:44.617845 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:44.617845 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:44.617845 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:44.617845 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:44.617845 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:44.617845 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:44.617845 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:44.617845 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:44.619190 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:44.619190 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:44.619190 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:18:44.619190 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:18:44.619190 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:18:44.619190 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:18:44.619190 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:18:44.619190 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:18:44.619190 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 17:18:44.619190 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:18:44.619190 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 408/801] 10.222.0.102 () {56 vars in 1122 bytes} [Thu Sep 1 17:18:44 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:18:44.619190 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82683d50-ba3a-4db2-b5bd-13b326d5a836 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:44.659039 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:44.663799 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:44.691918 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b256af79-610d-4978-b159-85170eb68ce8 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:44.742311 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acd3f0c-12e4-4ba2-8c0c-e7046a8ff68a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:44.743133 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82683d50-ba3a-4db2-b5bd-13b326d5a836 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:44.743259 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acd3f0c-12e4-4ba2-8c0c-e7046a8ff68a None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:44.743583 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acd3f0c-12e4-4ba2-8c0c-e7046a8ff68a None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:44.743956 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acd3f0c-12e4-4ba2-8c0c-e7046a8ff68a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:44.758213 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acd3f0c-12e4-4ba2-8c0c-e7046a8ff68a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:18:44.765518 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-0acd3f0c-12e4-4ba2-8c0c-e7046a8ff68a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:18:44.765518 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:44.765518 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:44.765518 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:44.765518 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:44.765518 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:44.765518 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:44.765518 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:44.765518 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:44.765518 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:44.765518 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:44.765518 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:44.765518 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:18:44.765518 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:18:44.765518 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:18:44.765518 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:18:44.766452 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:18:44.766452 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:18:44.766452 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:18:44.766452 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:18:44.766452 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:44.766452 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:44.766452 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:44.766452 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:44.766452 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:44.766452 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:44.766452 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:44.766452 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:44.766452 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:44.766452 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:44.766452 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:44.766452 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:44.767266 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:44.767266 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:44.767266 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:44.767266 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:44.767266 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:18:44.767266 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:18:44.767266 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:18:44.767266 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:18:44.767266 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:18:44.767266 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:18:44.767266 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:18:44.767266 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:18:44.768043 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 394/802] 10.222.0.102 () {56 vars in 1101 bytes} [Thu Sep 1 17:18:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:44.770344 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acd3f0c-12e4-4ba2-8c0c-e7046a8ff68a None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:44.782356 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604e0edf-0c40-49af-b41e-43a98fb42060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:44.782988 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604e0edf-0c40-49af-b41e-43a98fb42060 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:44.783264 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604e0edf-0c40-49af-b41e-43a98fb42060 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:44.783537 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604e0edf-0c40-49af-b41e-43a98fb42060 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:44.793179 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604e0edf-0c40-49af-b41e-43a98fb42060 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:44.794974 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604e0edf-0c40-49af-b41e-43a98fb42060 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:44.810421 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-604e0edf-0c40-49af-b41e-43a98fb42060 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:18:44.810421 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:44.810421 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:44.810421 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:44.810421 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:44.810421 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:44.810421 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:44.810421 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:44.810421 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:44.810421 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:44.810421 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:44.810421 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:44.810421 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:18:44.810421 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:18:44.810421 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:18:44.812181 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:18:44.812181 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:44.812181 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:44.812181 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:44.812181 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:44.812181 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:44.812181 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:44.812181 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:44.812181 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:44.812181 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:44.812181 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:44.812181 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:44.812181 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:44.812181 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:44.812181 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:44.812181 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:44.812181 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:44.814130 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:18:44.814130 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:18:44.814130 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:18:44.814130 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:18:44.814130 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:18:44.814130 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:18:44.814130 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:18:44.814130 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:18:44.817339 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 409/803] 10.222.0.102 () {56 vars in 1116 bytes} [Thu Sep 1 17:18:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:44.818163 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604e0edf-0c40-49af-b41e-43a98fb42060 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:44.914797 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acd3f0c-12e4-4ba2-8c0c-e7046a8ff68a None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:44.936328 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:44.944103 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604e0edf-0c40-49af-b41e-43a98fb42060 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:44.951963 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a25238-5f6e-4d08-a9b7-489fbf7b45b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:44.953005 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a25238-5f6e-4d08-a9b7-489fbf7b45b5 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:44.953290 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a25238-5f6e-4d08-a9b7-489fbf7b45b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:44.953593 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a25238-5f6e-4d08-a9b7-489fbf7b45b5 None admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:44.954327 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a25238-5f6e-4d08-a9b7-489fbf7b45b5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:44.954637 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a25238-5f6e-4d08-a9b7-489fbf7b45b5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:44.956971 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a25238-5f6e-4d08-a9b7-489fbf7b45b5 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:44.968181 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:44.977346 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 395/804] 10.222.0.102 () {56 vars in 1154 bytes} [Thu Sep 1 17:18:44 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 386 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:44.978175 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a25238-5f6e-4d08-a9b7-489fbf7b45b5 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:44.987244 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321b84e0-fc4a-4961-bf75-a538135952b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:44.987899 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321b84e0-fc4a-4961-bf75-a538135952b1 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:44.988187 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321b84e0-fc4a-4961-bf75-a538135952b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:44.988464 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321b84e0-fc4a-4961-bf75-a538135952b1 None admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:44.989131 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321b84e0-fc4a-4961-bf75-a538135952b1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:44.989538 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321b84e0-fc4a-4961-bf75-a538135952b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:44.990911 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321b84e0-fc4a-4961-bf75-a538135952b1 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:45.000303 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 410/805] 10.222.0.102 () {56 vars in 1104 bytes} [Thu Sep 1 17:18:44 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:45.001042 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321b84e0-fc4a-4961-bf75-a538135952b1 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:45.087741 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a25238-5f6e-4d08-a9b7-489fbf7b45b5 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:45.102710 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321b84e0-fc4a-4961-bf75-a538135952b1 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:45.121145 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf4c79c-0ec5-4a7b-a67a-733304e048fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:45.121786 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf4c79c-0ec5-4a7b-a67a-733304e048fd None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:45.122069 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf4c79c-0ec5-4a7b-a67a-733304e048fd None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:45.122346 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf4c79c-0ec5-4a7b-a67a-733304e048fd None admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:45.123042 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf4c79c-0ec5-4a7b-a67a-733304e048fd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:45.123349 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf4c79c-0ec5-4a7b-a67a-733304e048fd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:45.124785 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf4c79c-0ec5-4a7b-a67a-733304e048fd None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:45.136637 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 396/806] 10.222.0.102 () {56 vars in 1153 bytes} [Thu Sep 1 17:18:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:45.137468 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf4c79c-0ec5-4a7b-a67a-733304e048fd None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:45.137577 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476560d5-8fdb-441e-9508-05f1fb176bb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:45.138195 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476560d5-8fdb-441e-9508-05f1fb176bb2 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:45.138478 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476560d5-8fdb-441e-9508-05f1fb176bb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:45.138753 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476560d5-8fdb-441e-9508-05f1fb176bb2 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:45.139437 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476560d5-8fdb-441e-9508-05f1fb176bb2 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:45.140797 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476560d5-8fdb-441e-9508-05f1fb176bb2 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:45.149834 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-476560d5-8fdb-441e-9508-05f1fb176bb2 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 17:18:45.149834 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:45.149834 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 17:18:45.149834 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 17:18:45.149834 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 17:18:45.149834 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 17:18:45.149834 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 17:18:45.149834 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:18:45.149834 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 17:18:45.149834 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:18:45.149834 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 17:18:45.149834 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:18:45.149834 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:45.149834 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:45.149834 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:45.149834 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:45.149834 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:45.150847 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:45.150847 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:45.150847 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:45.150847 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:45.150847 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:45.150847 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:45.150847 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 17:18:45.150847 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 17:18:45.150847 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 17:18:45.150847 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 17:18:45.150847 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:45.150847 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:45.150847 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:45.150847 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:45.150847 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:45.150847 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:45.151661 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:45.151661 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:45.151661 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:45.151661 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:45.151661 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:45.151661 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:45.151661 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:45.151661 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:45.151661 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:45.151661 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:45.151661 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 17:18:45.151661 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:18:45.151661 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 17:18:45.151661 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:18:45.151661 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 411/807] 10.222.0.102 () {56 vars in 1134 bytes} [Thu Sep 1 17:18:45 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:45.153011 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476560d5-8fdb-441e-9508-05f1fb176bb2 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:45.177055 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:45.197935 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:45.239297 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf4c79c-0ec5-4a7b-a67a-733304e048fd None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:45.255108 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476560d5-8fdb-441e-9508-05f1fb176bb2 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:45.272969 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1f2b76-26b1-44d1-8244-b51d08855f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:45.273631 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1f2b76-26b1-44d1-8244-b51d08855f3d None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:45.273916 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1f2b76-26b1-44d1-8244-b51d08855f3d None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:45.274194 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1f2b76-26b1-44d1-8244-b51d08855f3d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:45.284715 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1f2b76-26b1-44d1-8244-b51d08855f3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:45.286239 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1f2b76-26b1-44d1-8244-b51d08855f3d None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:45.287644 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5f3e08-0abf-49b8-b501-4b2314e98e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:45.288312 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5f3e08-0abf-49b8-b501-4b2314e98e41 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:45.288583 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5f3e08-0abf-49b8-b501-4b2314e98e41 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:45.288859 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5f3e08-0abf-49b8-b501-4b2314e98e41 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:45.298561 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5f3e08-0abf-49b8-b501-4b2314e98e41 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:45.300487 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5f3e08-0abf-49b8-b501-4b2314e98e41 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:45.300725 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-2e1f2b76-26b1-44d1-8244-b51d08855f3d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:45.300725 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:45.300725 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:45.300725 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:45.300725 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:45.300725 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:45.300725 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:45.300725 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:45.300725 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:45.300725 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:45.300725 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:45.300725 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:45.300725 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:18:45.300725 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:18:45.300725 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:18:45.300725 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:18:45.301807 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:45.301807 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:45.301807 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:45.301807 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:45.301807 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:45.301807 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:45.301807 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:45.301807 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:45.301807 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:45.301807 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:45.301807 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:45.301807 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:45.301807 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:45.301807 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:45.301807 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:45.301807 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:45.301807 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:18:45.303000 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:18:45.303000 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:45.303000 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:18:45.303000 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 397/808] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:18:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:45.303269 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c285a9de-c5ad-45ab-87be-434e56474f8a None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:45.303358 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c285a9de-c5ad-45ab-87be-434e56474f8a None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:45.303906 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c285a9de-c5ad-45ab-87be-434e56474f8a None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:45.305192 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 398/809] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:18:45 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:18:45.305601 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c285a9de-c5ad-45ab-87be-434e56474f8a None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:45.323441 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-4d5f3e08-0abf-49b8-b501-4b2314e98e41 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:45.323441 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:45.323441 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:45.323441 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:45.323441 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:45.323441 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:45.323441 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:45.323441 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:45.323441 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:45.323441 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:45.323441 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:45.323441 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:45.323441 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:18:45.323441 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:18:45.323441 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:18:45.323441 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:18:45.324435 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:45.324435 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:45.324435 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:45.324435 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:45.324435 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:45.324435 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:45.324435 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:45.324435 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:45.324435 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:45.324435 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:45.324435 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:45.324435 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:45.324435 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:45.324435 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:45.324435 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:45.324435 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:45.324435 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:18:45.325387 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:18:45.325387 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:45.325387 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:18:45.325387 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 412/810] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:18:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:45.325387 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5f3e08-0abf-49b8-b501-4b2314e98e41 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:45.411123 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c285a9de-c5ad-45ab-87be-434e56474f8a None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:45.412769 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:45.421954 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:45.439723 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5f3e08-0abf-49b8-b501-4b2314e98e41 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:45.445949 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb114f84-9e21-4b49-bf0b-a5b145c260aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:45.446613 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb114f84-9e21-4b49-bf0b-a5b145c260aa None admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:45.446900 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb114f84-9e21-4b49-bf0b-a5b145c260aa None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:45.447174 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb114f84-9e21-4b49-bf0b-a5b145c260aa None admin] PATH_INFO: `/v3/projects/e78fff7f1f6e4f68b8d233ad2e5bbd4f/users/d55edf6cd1804006aa8269b4c6f80aec/roles/36577a6f46d642d4b81805ede9872490` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:45.449097 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb114f84-9e21-4b49-bf0b-a5b145c260aa None admin] RBAC: Authorizing `identity:create_grant(project_id=e78fff7f1f6e4f68b8d233ad2e5bbd4f, user_id=d55edf6cd1804006aa8269b4c6f80aec, role_id=36577a6f46d642d4b81805ede9872490)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:45.450633 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb114f84-9e21-4b49-bf0b-a5b145c260aa None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:45.463270 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 399/811] 10.222.0.102 () {58 vars in 1444 bytes} [Thu Sep 1 17:18:45 2022] PUT /identity/v3/projects/e78fff7f1f6e4f68b8d233ad2e5bbd4f/users/d55edf6cd1804006aa8269b4c6f80aec/roles/36577a6f46d642d4b81805ede9872490 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:18:45.464090 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb114f84-9e21-4b49-bf0b-a5b145c260aa None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:45.475657 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ef4f03-ea80-4e38-9e9e-e5a0d6d629fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:45.476321 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ef4f03-ea80-4e38-9e9e-e5a0d6d629fd None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:45.476657 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ef4f03-ea80-4e38-9e9e-e5a0d6d629fd None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:45.476939 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ef4f03-ea80-4e38-9e9e-e5a0d6d629fd None admin] PATH_INFO: `/v3/services` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:45.477624 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ef4f03-ea80-4e38-9e9e-e5a0d6d629fd None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:45.478187 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ef4f03-ea80-4e38-9e9e-e5a0d6d629fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:45.479605 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ef4f03-ea80-4e38-9e9e-e5a0d6d629fd None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:45.486444 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 413/812] 10.222.0.102 () {56 vars in 1129 bytes} [Thu Sep 1 17:18:45 2022] GET /identity/v3/services?name=block-storage => generated 131 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:45.488261 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c114bce0-56ec-4fb7-8111-76b4a3913c25 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:45.488553 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c114bce0-56ec-4fb7-8111-76b4a3913c25 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:45.488825 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c114bce0-56ec-4fb7-8111-76b4a3913c25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:45.509944 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-c114bce0-56ec-4fb7-8111-76b4a3913c25 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:18:45.511191 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-c114bce0-56ec-4fb7-8111-76b4a3913c25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:18:45.589373 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 414/813] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:18:45 2022] POST /identity/v3/auth/tokens => generated 1003 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:18:45.589373 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c114bce0-56ec-4fb7-8111-76b4a3913c25 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:45.613394 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb114f84-9e21-4b49-bf0b-a5b145c260aa None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:45.656567 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ebcc3d-0ef7-4c8e-8019-938b7f3ec3d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:45.657187 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ebcc3d-0ef7-4c8e-8019-938b7f3ec3d3 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:45.657516 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ebcc3d-0ef7-4c8e-8019-938b7f3ec3d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:45.657790 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ebcc3d-0ef7-4c8e-8019-938b7f3ec3d3 None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:45.658916 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ebcc3d-0ef7-4c8e-8019-938b7f3ec3d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:45.659759 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ebcc3d-0ef7-4c8e-8019-938b7f3ec3d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:45.661171 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ebcc3d-0ef7-4c8e-8019-938b7f3ec3d3 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:45.675937 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 400/814] 10.222.0.102 () {56 vars in 1114 bytes} [Thu Sep 1 17:18:45 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:45.676784 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ebcc3d-0ef7-4c8e-8019-938b7f3ec3d3 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:45.685093 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:45.710055 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:45.735915 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c114bce0-56ec-4fb7-8111-76b4a3913c25 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:45.784315 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e35c40-a614-41c2-ab02-5ad3c28f033d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:45.785837 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e35c40-a614-41c2-ab02-5ad3c28f033d None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:45.786125 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e35c40-a614-41c2-ab02-5ad3c28f033d None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:45.786670 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e35c40-a614-41c2-ab02-5ad3c28f033d None admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:45.787326 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e35c40-a614-41c2-ab02-5ad3c28f033d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:45.787631 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e35c40-a614-41c2-ab02-5ad3c28f033d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:45.788731 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ebcc3d-0ef7-4c8e-8019-938b7f3ec3d3 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:45.789015 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e35c40-a614-41c2-ab02-5ad3c28f033d None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:45.804381 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 415/815] 10.222.0.102 () {56 vars in 1114 bytes} [Thu Sep 1 17:18:45 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:45.806078 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e35c40-a614-41c2-ab02-5ad3c28f033d None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:45.835537 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a1cd01-6890-4a6e-8049-847260f3360b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:45.836164 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a1cd01-6890-4a6e-8049-847260f3360b None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:45.836446 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a1cd01-6890-4a6e-8049-847260f3360b None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:45.836720 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a1cd01-6890-4a6e-8049-847260f3360b None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:45.837664 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a1cd01-6890-4a6e-8049-847260f3360b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:45.837977 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a1cd01-6890-4a6e-8049-847260f3360b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:45.839628 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a1cd01-6890-4a6e-8049-847260f3360b None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:45.852151 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 401/816] 10.222.0.102 () {56 vars in 1129 bytes} [Thu Sep 1 17:18:45 2022] GET /identity/v3/services?type=block-storage => generated 369 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:45.852151 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a1cd01-6890-4a6e-8049-847260f3360b None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:45.929861 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e35c40-a614-41c2-ab02-5ad3c28f033d None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:45.958269 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:45.963869 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:45.967131 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a1cd01-6890-4a6e-8049-847260f3360b None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:45.973147 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dff0d89-ecf1-4e3f-97ad-353fed648e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:45.973818 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dff0d89-ecf1-4e3f-97ad-353fed648e32 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:45.974101 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dff0d89-ecf1-4e3f-97ad-353fed648e32 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:45.974375 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dff0d89-ecf1-4e3f-97ad-353fed648e32 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:45.987387 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dff0d89-ecf1-4e3f-97ad-353fed648e32 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:45.988581 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dff0d89-ecf1-4e3f-97ad-353fed648e32 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:46.000586 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-2dff0d89-ecf1-4e3f-97ad-353fed648e32 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:18:46.000586 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:46.000586 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:46.000586 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:46.000586 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:46.000586 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:46.000586 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:46.000586 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:46.000586 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:46.000586 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:46.000586 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:46.000586 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:46.000586 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:18:46.000586 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:18:46.000586 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:18:46.001508 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:18:46.001508 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:46.001508 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:46.001508 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:46.001508 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:46.001508 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:46.001508 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:46.001508 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:46.001508 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:46.001508 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:46.001508 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:46.001508 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:46.001508 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:46.001508 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:46.001508 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:46.001508 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:46.001508 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:46.003030 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:18:46.003030 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:18:46.003030 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:18:46.003030 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:18:46.003030 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:18:46.003030 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:18:46.003030 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:18:46.003030 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:18:46.003030 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 416/817] 10.222.0.102 () {56 vars in 1116 bytes} [Thu Sep 1 17:18:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:46.003030 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dff0d89-ecf1-4e3f-97ad-353fed648e32 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:46.012632 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd42045-7126-46f7-97df-50a72395a8f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:46.013275 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd42045-7126-46f7-97df-50a72395a8f0 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:46.013584 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd42045-7126-46f7-97df-50a72395a8f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:46.013858 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd42045-7126-46f7-97df-50a72395a8f0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:46.021774 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd42045-7126-46f7-97df-50a72395a8f0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:46.023220 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd42045-7126-46f7-97df-50a72395a8f0 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:46.035343 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-7dd42045-7126-46f7-97df-50a72395a8f0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 17:18:46.035343 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:46.035343 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:46.035343 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:46.035343 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:46.035343 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:46.035343 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:46.035343 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:46.035343 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:46.035343 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:46.035343 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:46.035343 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:46.035343 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:18:46.035343 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:18:46.035343 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:18:46.035343 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:18:46.036448 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:46.036448 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:46.036448 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:18:46.036448 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:18:46.036448 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:18:46.036448 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:18:46.036448 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:46.036448 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:46.036448 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:46.036448 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:46.036448 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:46.036448 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:46.036448 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:46.036448 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:46.036448 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:46.036448 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:46.036448 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:46.036448 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:46.037782 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:46.037782 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:46.037782 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:18:46.037782 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:18:46.037782 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:18:46.037782 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:18:46.037782 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:18:46.037782 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:18:46.037782 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 17:18:46.037782 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:18:46.037782 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 402/818] 10.222.0.102 () {56 vars in 1098 bytes} [Thu Sep 1 17:18:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:46.037782 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd42045-7126-46f7-97df-50a72395a8f0 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:46.116957 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dff0d89-ecf1-4e3f-97ad-353fed648e32 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:46.148498 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd42045-7126-46f7-97df-50a72395a8f0 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:46.152107 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96bcd20-ae92-4ed9-ae88-a5478032d06d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:46.152726 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96bcd20-ae92-4ed9-ae88-a5478032d06d None admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:46.153007 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96bcd20-ae92-4ed9-ae88-a5478032d06d None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:46.153280 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96bcd20-ae92-4ed9-ae88-a5478032d06d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:46.153976 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96bcd20-ae92-4ed9-ae88-a5478032d06d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:46.155393 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96bcd20-ae92-4ed9-ae88-a5478032d06d None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:46.178456 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 417/819] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:18:46 2022] POST /identity/v3/endpoints => generated 348 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:18:46.179718 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96bcd20-ae92-4ed9-ae88-a5478032d06d None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:46.198522 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2a16db-c244-43ef-b769-a8634c8c0a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:46.199184 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2a16db-c244-43ef-b769-a8634c8c0a2e None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:46.199469 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2a16db-c244-43ef-b769-a8634c8c0a2e None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:46.199945 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2a16db-c244-43ef-b769-a8634c8c0a2e None admin] PATH_INFO: `/v3/services/network` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:46.200741 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2a16db-c244-43ef-b769-a8634c8c0a2e None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:46.202589 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2a16db-c244-43ef-b769-a8634c8c0a2e None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:46.216288 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-4b2a16db-c244-43ef-b769-a8634c8c0a2e None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 17:18:46.216288 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:46.216288 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 17:18:46.216288 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 17:18:46.216288 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 17:18:46.216288 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 17:18:46.216288 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 17:18:46.216288 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:18:46.216288 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 17:18:46.216288 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:18:46.216288 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 17:18:46.216288 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:18:46.216288 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:46.216288 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:46.216288 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:46.216288 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:46.216288 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:46.217404 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:46.217404 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:46.217404 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:46.217404 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:46.217404 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:46.217404 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:46.217404 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 17:18:46.217404 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 17:18:46.217404 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 17:18:46.217404 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 17:18:46.217404 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:46.217404 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:46.217404 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:46.217404 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:46.217404 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:46.217404 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:46.218261 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:46.218261 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:46.218261 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:46.218261 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:46.218261 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:46.218261 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:46.218261 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:46.218261 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:46.218261 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:46.218261 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:46.218261 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 17:18:46.218261 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:18:46.218261 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 17:18:46.218261 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:18:46.221468 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 403/820] 10.222.0.102 () {56 vars in 1116 bytes} [Thu Sep 1 17:18:46 2022] GET /identity/v3/services/network => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:46.222673 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2a16db-c244-43ef-b769-a8634c8c0a2e None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:46.248613 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:46.251729 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:46.322216 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96bcd20-ae92-4ed9-ae88-a5478032d06d None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:46.351957 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2a16db-c244-43ef-b769-a8634c8c0a2e None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:46.362654 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a6564a-2e99-4e93-8bbc-943fd6377300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:46.363281 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a6564a-2e99-4e93-8bbc-943fd6377300 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:46.363563 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a6564a-2e99-4e93-8bbc-943fd6377300 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:46.363840 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a6564a-2e99-4e93-8bbc-943fd6377300 None admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:46.364724 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a6564a-2e99-4e93-8bbc-943fd6377300 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:46.365041 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a6564a-2e99-4e93-8bbc-943fd6377300 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:46.366637 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a6564a-2e99-4e93-8bbc-943fd6377300 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:46.381264 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 418/821] 10.222.0.102 () {56 vars in 1153 bytes} [Thu Sep 1 17:18:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:46.382456 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a6564a-2e99-4e93-8bbc-943fd6377300 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:46.395445 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46899617-e351-4411-a516-2248a4ae77fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:46.396892 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46899617-e351-4411-a516-2248a4ae77fa None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:46.397398 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46899617-e351-4411-a516-2248a4ae77fa None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:46.397834 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46899617-e351-4411-a516-2248a4ae77fa None admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:46.398582 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46899617-e351-4411-a516-2248a4ae77fa None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:46.398892 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46899617-e351-4411-a516-2248a4ae77fa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:46.400564 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46899617-e351-4411-a516-2248a4ae77fa None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:46.419806 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 404/822] 10.222.0.102 () {56 vars in 1138 bytes} [Thu Sep 1 17:18:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 370 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:46.420645 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46899617-e351-4411-a516-2248a4ae77fa None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:46.495587 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a6564a-2e99-4e93-8bbc-943fd6377300 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:46.498893 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:46.506377 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:46.536122 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33561b3d-bbfe-4ac6-8691-82eb0bccaee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:46.536897 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33561b3d-bbfe-4ac6-8691-82eb0bccaee3 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:46.537357 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33561b3d-bbfe-4ac6-8691-82eb0bccaee3 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:46.537738 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33561b3d-bbfe-4ac6-8691-82eb0bccaee3 None admin] PATH_INFO: `/v3/services` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:46.538545 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33561b3d-bbfe-4ac6-8691-82eb0bccaee3 None admin] RBAC: Adding query filter params (name=network) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:46.538950 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33561b3d-bbfe-4ac6-8691-82eb0bccaee3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:46.540563 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33561b3d-bbfe-4ac6-8691-82eb0bccaee3 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:46.546121 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 419/823] 10.222.0.102 () {56 vars in 1117 bytes} [Thu Sep 1 17:18:46 2022] GET /identity/v3/services?name=network => generated 125 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:46.547174 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33561b3d-bbfe-4ac6-8691-82eb0bccaee3 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:46.551248 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46899617-e351-4411-a516-2248a4ae77fa None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:46.604729 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0abd84-35b0-4a83-9f2f-2114dbd2b123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:46.605826 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0abd84-35b0-4a83-9f2f-2114dbd2b123 None admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:46.606253 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0abd84-35b0-4a83-9f2f-2114dbd2b123 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:46.606691 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0abd84-35b0-4a83-9f2f-2114dbd2b123 None admin] PATH_INFO: `/v3/projects/e78fff7f1f6e4f68b8d233ad2e5bbd4f/users/575a76b99ea945efb1024fbb3dc81aaf/roles/f21e23654e4740158c7d4f46e726e515` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:46.625422 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0abd84-35b0-4a83-9f2f-2114dbd2b123 None admin] RBAC: Authorizing `identity:create_grant(project_id=e78fff7f1f6e4f68b8d233ad2e5bbd4f, user_id=575a76b99ea945efb1024fbb3dc81aaf, role_id=f21e23654e4740158c7d4f46e726e515)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:46.627212 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0abd84-35b0-4a83-9f2f-2114dbd2b123 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:46.645962 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 405/824] 10.222.0.102 () {58 vars in 1444 bytes} [Thu Sep 1 17:18:46 2022] PUT /identity/v3/projects/e78fff7f1f6e4f68b8d233ad2e5bbd4f/users/575a76b99ea945efb1024fbb3dc81aaf/roles/f21e23654e4740158c7d4f46e726e515 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:18:46.647145 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0abd84-35b0-4a83-9f2f-2114dbd2b123 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:46.727618 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33561b3d-bbfe-4ac6-8691-82eb0bccaee3 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:46.774684 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0abd84-35b0-4a83-9f2f-2114dbd2b123 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:46.814488 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f57312-b9c8-48dd-8f0b-531358cb61e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:46.814789 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f57312-b9c8-48dd-8f0b-531358cb61e1 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:46.815071 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f57312-b9c8-48dd-8f0b-531358cb61e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:46.815346 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f57312-b9c8-48dd-8f0b-531358cb61e1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:46.819647 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:46.827666 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d46b7d-1992-4e69-9cd5-653cb24fc856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:46.828772 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d46b7d-1992-4e69-9cd5-653cb24fc856 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:46.829258 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d46b7d-1992-4e69-9cd5-653cb24fc856 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:46.829762 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d46b7d-1992-4e69-9cd5-653cb24fc856 None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:46.830629 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d46b7d-1992-4e69-9cd5-653cb24fc856 None admin] RBAC: Adding query filter params (type=network) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:46.833511 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d46b7d-1992-4e69-9cd5-653cb24fc856 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:46.835350 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d46b7d-1992-4e69-9cd5-653cb24fc856 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:46.842983 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 406/825] 10.222.0.102 () {56 vars in 1117 bytes} [Thu Sep 1 17:18:46 2022] GET /identity/v3/services?type=network => generated 352 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:46.855156 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d46b7d-1992-4e69-9cd5-653cb24fc856 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:46.892285 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:46.933437 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f57312-b9c8-48dd-8f0b-531358cb61e1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:46.934902 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f57312-b9c8-48dd-8f0b-531358cb61e1 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:46.971516 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d46b7d-1992-4e69-9cd5-653cb24fc856 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:47.010743 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da9b9bd-2e08-43fc-afee-62be82192113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:47.011827 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da9b9bd-2e08-43fc-afee-62be82192113 None admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:47.012341 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da9b9bd-2e08-43fc-afee-62be82192113 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:47.012815 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da9b9bd-2e08-43fc-afee-62be82192113 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:47.013840 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da9b9bd-2e08-43fc-afee-62be82192113 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:47.015655 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da9b9bd-2e08-43fc-afee-62be82192113 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:47.049938 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 407/826] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:18:46 2022] POST /identity/v3/endpoints => generated 339 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:18:47.064072 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 420/827] 10.222.0.102 () {58 vars in 1287 bytes} [Thu Sep 1 17:18:46 2022] GET /identity/v3/auth/tokens => generated 1003 bytes in 517 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 17:18:47.070666 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da9b9bd-2e08-43fc-afee-62be82192113 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:47.134616 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:47.221554 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da9b9bd-2e08-43fc-afee-62be82192113 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:47.260212 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5978e175-d5b2-421b-ac9a-a441abe5d14c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:47.261283 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5978e175-d5b2-421b-ac9a-a441abe5d14c None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:47.261932 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5978e175-d5b2-421b-ac9a-a441abe5d14c None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:47.262431 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5978e175-d5b2-421b-ac9a-a441abe5d14c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:47.275564 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5978e175-d5b2-421b-ac9a-a441abe5d14c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:47.277447 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5978e175-d5b2-421b-ac9a-a441abe5d14c None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:47.293967 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-5978e175-d5b2-421b-ac9a-a441abe5d14c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:47.293967 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:47.293967 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:47.293967 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:47.293967 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:47.293967 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:47.293967 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:47.293967 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:47.293967 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:47.293967 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:47.293967 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:47.293967 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:47.293967 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:18:47.293967 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:18:47.293967 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:18:47.293967 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:18:47.294961 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:47.294961 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:47.294961 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:47.294961 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:47.294961 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:47.294961 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:47.294961 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:47.294961 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:47.294961 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:47.294961 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:47.294961 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:47.294961 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:47.294961 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:47.294961 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:47.294961 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:47.294961 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:47.294961 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:18:47.295839 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:18:47.295839 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:47.295839 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:18:47.298104 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 408/828] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:18:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:47.305476 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f57312-b9c8-48dd-8f0b-531358cb61e1 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:47.380975 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:47.418956 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f57312-b9c8-48dd-8f0b-531358cb61e1 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:47.459955 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064bd94e-b2a1-4077-8fd7-20aa754a37c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:47.460658 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064bd94e-b2a1-4077-8fd7-20aa754a37c0 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:47.460947 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064bd94e-b2a1-4077-8fd7-20aa754a37c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:47.461225 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064bd94e-b2a1-4077-8fd7-20aa754a37c0 None admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:47.461910 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064bd94e-b2a1-4077-8fd7-20aa754a37c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:47.462317 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064bd94e-b2a1-4077-8fd7-20aa754a37c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:47.463692 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064bd94e-b2a1-4077-8fd7-20aa754a37c0 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:47.476331 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 421/829] 10.222.0.102 () {56 vars in 1114 bytes} [Thu Sep 1 17:18:47 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:47.481773 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5978e175-d5b2-421b-ac9a-a441abe5d14c None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:47.597061 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:47.609851 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5978e175-d5b2-421b-ac9a-a441abe5d14c None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:47.614877 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fe46e6-e262-489f-9995-0f18529df282 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:47.615173 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fe46e6-e262-489f-9995-0f18529df282 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:47.615449 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fe46e6-e262-489f-9995-0f18529df282 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:47.616917 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 422/830] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:18:47 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:18:47.622593 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b294aa7-8928-4752-970f-3ca8915a282b None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:47.622886 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b294aa7-8928-4752-970f-3ca8915a282b None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:47.623165 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b294aa7-8928-4752-970f-3ca8915a282b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:47.649515 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-0b294aa7-8928-4752-970f-3ca8915a282b None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:18:47.651042 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-0b294aa7-8928-4752-970f-3ca8915a282b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:18:47.658565 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ebea6e-c246-4c0c-b1f7-7fdaa14c4ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:47.659975 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ebea6e-c246-4c0c-b1f7-7fdaa14c4ca1 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:47.660466 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ebea6e-c246-4c0c-b1f7-7fdaa14c4ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:47.661025 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ebea6e-c246-4c0c-b1f7-7fdaa14c4ca1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:47.671336 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ebea6e-c246-4c0c-b1f7-7fdaa14c4ca1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:47.673139 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ebea6e-c246-4c0c-b1f7-7fdaa14c4ca1 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:47.688545 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-80ebea6e-c246-4c0c-b1f7-7fdaa14c4ca1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:18:47.688545 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:47.688545 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:47.688545 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:47.688545 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:47.688545 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:47.688545 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:47.688545 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:47.688545 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:47.688545 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:47.688545 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:47.688545 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:47.688545 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:18:47.688545 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:18:47.688545 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:18:47.689510 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:18:47.689510 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:47.689510 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:47.689510 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:47.689510 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:47.689510 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:47.689510 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:47.689510 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:47.689510 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:47.689510 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:47.689510 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:47.689510 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:47.689510 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:47.689510 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:47.689510 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:47.689510 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:47.689510 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:47.690419 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:18:47.690419 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:18:47.690419 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:18:47.690419 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:18:47.690419 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:18:47.690419 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:18:47.690419 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:18:47.690419 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:18:47.692875 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 409/831] 10.222.0.102 () {56 vars in 1116 bytes} [Thu Sep 1 17:18:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:47.698782 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ebea6e-c246-4c0c-b1f7-7fdaa14c4ca1 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:47.743654 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 423/832] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:18:47 2022] POST /identity/v3/auth/tokens => generated 1163 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 17:18:47.810806 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ebea6e-c246-4c0c-b1f7-7fdaa14c4ca1 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:47.846947 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556d3e63-c661-499c-937d-eb028e9310c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:47.847653 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556d3e63-c661-499c-937d-eb028e9310c2 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:47.847870 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556d3e63-c661-499c-937d-eb028e9310c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:47.850248 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556d3e63-c661-499c-937d-eb028e9310c2 None admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:47.850248 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556d3e63-c661-499c-937d-eb028e9310c2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:47.850248 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556d3e63-c661-499c-937d-eb028e9310c2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:47.854043 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556d3e63-c661-499c-937d-eb028e9310c2 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:47.864691 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 410/833] 10.222.0.102 () {56 vars in 1153 bytes} [Thu Sep 1 17:18:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:18:47.874420 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b294aa7-8928-4752-970f-3ca8915a282b None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:47.943227 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:47.986773 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b294aa7-8928-4752-970f-3ca8915a282b None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:48.023242 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff415442-42b8-4f32-890f-c65d8bff65f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:48.023857 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff415442-42b8-4f32-890f-c65d8bff65f7 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:48.024139 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff415442-42b8-4f32-890f-c65d8bff65f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:48.024415 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff415442-42b8-4f32-890f-c65d8bff65f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:48.025275 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff415442-42b8-4f32-890f-c65d8bff65f7 None admin] RBAC: Adding query filter params (role.id=36577a6f46d642d4b81805ede9872490, scope.project.id=e78fff7f1f6e4f68b8d233ad2e5bbd4f, user.id=5f8293316baf4774b62fc11d55d50865) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:48.025630 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff415442-42b8-4f32-890f-c65d8bff65f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:48.027226 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff415442-42b8-4f32-890f-c65d8bff65f7 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:48.033651 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 424/834] 10.222.0.102 () {56 vars in 1379 bytes} [Thu Sep 1 17:18:47 2022] GET /identity/v3/role_assignments?user.id=5f8293316baf4774b62fc11d55d50865&scope.project.id=e78fff7f1f6e4f68b8d233ad2e5bbd4f&role.id=36577a6f46d642d4b81805ede9872490 => generated 610 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:48.063902 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556d3e63-c661-499c-937d-eb028e9310c2 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:48.078199 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:48.180484 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556d3e63-c661-499c-937d-eb028e9310c2 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:48.217461 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6c60f8-e4d0-4dd6-bef4-046b6b82a891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:48.218112 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6c60f8-e4d0-4dd6-bef4-046b6b82a891 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:48.218391 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6c60f8-e4d0-4dd6-bef4-046b6b82a891 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:48.218662 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6c60f8-e4d0-4dd6-bef4-046b6b82a891 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:48.230711 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6c60f8-e4d0-4dd6-bef4-046b6b82a891 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:18:48.235496 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-be6c60f8-e4d0-4dd6-bef4-046b6b82a891 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:18:48.235496 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:48.235496 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:48.235496 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:48.235496 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:48.235496 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:48.235496 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:48.235496 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:48.235496 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:48.235496 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:48.235496 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:48.235496 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:48.235496 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:18:48.235496 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:18:48.235496 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:18:48.235496 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:18:48.236389 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:18:48.236389 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:18:48.236389 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:18:48.236389 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:18:48.236389 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:48.236389 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:48.236389 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:48.236389 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:48.236389 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:48.236389 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:48.236389 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:48.236389 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:48.236389 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:48.236389 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:48.236389 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:48.236389 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:48.237227 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:48.237227 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:48.237227 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:48.237227 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:48.237227 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:18:48.237227 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:18:48.237227 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:18:48.237227 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:18:48.237227 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:18:48.237227 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:18:48.237227 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:18:48.237227 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:18:48.241382 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 411/835] 10.222.0.102 () {56 vars in 1101 bytes} [Thu Sep 1 17:18:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:48.243086 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff415442-42b8-4f32-890f-c65d8bff65f7 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:48.352503 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff415442-42b8-4f32-890f-c65d8bff65f7 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:48.369993 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999d3e3f-57e9-4daf-b9d4-24a77bbd2283 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:48.370664 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999d3e3f-57e9-4daf-b9d4-24a77bbd2283 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:48.371139 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999d3e3f-57e9-4daf-b9d4-24a77bbd2283 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:48.373563 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 412/836] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:18:48 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Sep 01 17:18:48.377627 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:48.379627 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461c0855-2621-4075-981e-fa15532baf1c None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:48.379962 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461c0855-2621-4075-981e-fa15532baf1c None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:48.380282 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461c0855-2621-4075-981e-fa15532baf1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:48.391470 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16825394-80c2-48ef-9005-b4b42f6ad048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:48.392077 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16825394-80c2-48ef-9005-b4b42f6ad048 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:48.392355 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16825394-80c2-48ef-9005-b4b42f6ad048 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:48.392621 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16825394-80c2-48ef-9005-b4b42f6ad048 None admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:48.393285 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16825394-80c2-48ef-9005-b4b42f6ad048 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:48.394339 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16825394-80c2-48ef-9005-b4b42f6ad048 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:48.395715 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16825394-80c2-48ef-9005-b4b42f6ad048 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:48.409071 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 425/837] 10.222.0.102 () {56 vars in 1104 bytes} [Thu Sep 1 17:18:48 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:48.410988 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16825394-80c2-48ef-9005-b4b42f6ad048 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:48.411368 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-461c0855-2621-4075-981e-fa15532baf1c None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:18:48.412818 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-461c0855-2621-4075-981e-fa15532baf1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:18:48.510558 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:48.520112 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 413/838] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:18:48 2022] POST /identity/v3/auth/tokens => generated 1163 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:18:48.537150 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16825394-80c2-48ef-9005-b4b42f6ad048 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:48.573646 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d799834a-37b5-4ddd-bfc6-b56d288ffb6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:48.574617 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d799834a-37b5-4ddd-bfc6-b56d288ffb6c None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:48.575081 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d799834a-37b5-4ddd-bfc6-b56d288ffb6c None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:48.575534 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d799834a-37b5-4ddd-bfc6-b56d288ffb6c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:48.587680 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d799834a-37b5-4ddd-bfc6-b56d288ffb6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:48.589429 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d799834a-37b5-4ddd-bfc6-b56d288ffb6c None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:48.604464 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-d799834a-37b5-4ddd-bfc6-b56d288ffb6c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:48.604464 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:48.604464 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:48.604464 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:48.604464 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:48.604464 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:48.604464 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:48.604464 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:48.604464 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:48.604464 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:48.604464 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:48.604464 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:48.604464 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:18:48.604464 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:18:48.604464 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:18:48.604464 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:18:48.605490 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:48.605490 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:48.605490 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:48.605490 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:48.605490 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:48.605490 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:48.605490 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:48.605490 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:48.605490 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:48.605490 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:48.605490 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:48.605490 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:48.605490 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:48.605490 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:48.605490 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:48.605490 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:48.605490 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:18:48.606389 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:18:48.606389 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:48.606389 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:18:48.612756 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 426/839] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:18:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:18:48.614276 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461c0855-2621-4075-981e-fa15532baf1c None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:48.721102 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461c0855-2621-4075-981e-fa15532baf1c None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:48.757846 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1538f2bc-2039-4646-afa1-5805e96d9db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:48.758698 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1538f2bc-2039-4646-afa1-5805e96d9db9 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:48.759131 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1538f2bc-2039-4646-afa1-5805e96d9db9 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:48.759575 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1538f2bc-2039-4646-afa1-5805e96d9db9 None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:48.760379 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1538f2bc-2039-4646-afa1-5805e96d9db9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:48.760893 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1538f2bc-2039-4646-afa1-5805e96d9db9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:48.762436 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1538f2bc-2039-4646-afa1-5805e96d9db9 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:48.779414 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 414/840] 10.222.0.102 () {56 vars in 1114 bytes} [Thu Sep 1 17:18:48 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:48.780966 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d799834a-37b5-4ddd-bfc6-b56d288ffb6c None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:48.807099 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1538f2bc-2039-4646-afa1-5805e96d9db9 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:48.829143 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:48.883529 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:48.906435 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d799834a-37b5-4ddd-bfc6-b56d288ffb6c None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:48.940124 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1538f2bc-2039-4646-afa1-5805e96d9db9 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:48.949648 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e766e0c2-ccd3-435f-bf8d-449b4eb53122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:48.950282 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e766e0c2-ccd3-435f-bf8d-449b4eb53122 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:48.950563 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e766e0c2-ccd3-435f-bf8d-449b4eb53122 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:48.950839 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e766e0c2-ccd3-435f-bf8d-449b4eb53122 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:48.958771 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e766e0c2-ccd3-435f-bf8d-449b4eb53122 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:48.960214 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e766e0c2-ccd3-435f-bf8d-449b4eb53122 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:48.974055 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-e766e0c2-ccd3-435f-bf8d-449b4eb53122 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 01 17:18:48.974055 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:48.974055 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:48.974055 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:48.974055 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:48.974055 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:48.974055 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:48.974055 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:48.974055 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:48.974055 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:48.974055 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:48.974055 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:48.974055 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:18:48.974055 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:18:48.974055 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:18:48.974055 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:18:48.974995 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:48.974995 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:48.974995 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:18:48.974995 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:18:48.974995 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:18:48.974995 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:18:48.974995 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:48.974995 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:48.974995 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:48.974995 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:48.974995 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:48.974995 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:48.974995 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:48.974995 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:48.974995 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:48.974995 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:48.974995 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:48.974995 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:48.975991 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:48.975991 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:48.975991 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:18:48.975991 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:18:48.975991 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:18:48.975991 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:18:48.975991 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:18:48.975991 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:18:48.975991 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 01 17:18:48.975991 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:18:48.975991 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 427/841] 10.222.0.102 () {56 vars in 1101 bytes} [Thu Sep 1 17:18:48 2022] GET /identity/v3/users/swift => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:48.976575 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8a98c4-b990-4aa4-928f-7bc751d3323d None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:48.976575 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8a98c4-b990-4aa4-928f-7bc751d3323d None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:48.977367 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8a98c4-b990-4aa4-928f-7bc751d3323d None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:48.978500 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 428/842] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:18:48 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:18:48.982948 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8a98c4-b990-4aa4-928f-7bc751d3323d None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:48.986179 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de66e211-5eb8-40dc-a08c-a914724eb005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:48.987212 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de66e211-5eb8-40dc-a08c-a914724eb005 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:48.987212 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de66e211-5eb8-40dc-a08c-a914724eb005 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:48.987400 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de66e211-5eb8-40dc-a08c-a914724eb005 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:48.987976 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de66e211-5eb8-40dc-a08c-a914724eb005 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:48.989373 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de66e211-5eb8-40dc-a08c-a914724eb005 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:48.998695 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-de66e211-5eb8-40dc-a08c-a914724eb005 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 17:18:48.998695 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:48.998695 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 17:18:48.998695 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 17:18:48.998695 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 17:18:48.998695 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 17:18:48.998695 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 17:18:48.998695 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:18:48.998695 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 17:18:48.998695 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:18:48.998695 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 17:18:48.998695 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:18:48.998695 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:48.998695 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:48.998695 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:48.998695 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:48.998695 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:48.999711 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:48.999711 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:48.999711 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:48.999711 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:48.999711 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:48.999711 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:48.999711 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 17:18:48.999711 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 17:18:48.999711 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 17:18:48.999711 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 17:18:48.999711 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:48.999711 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:48.999711 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:48.999711 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:48.999711 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:48.999711 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:49.000537 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:49.000537 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:49.000537 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:49.000537 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:49.000537 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:49.000537 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:49.000537 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:49.000537 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:49.000537 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:49.000537 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:49.000537 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 17:18:49.000537 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:18:49.000537 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 17:18:49.000537 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:18:49.000537 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 415/843] 10.222.0.102 () {56 vars in 1119 bytes} [Thu Sep 1 17:18:48 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:49.002075 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49cb2b9-4fd7-448e-8e84-27912bd8159c None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:49.002075 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49cb2b9-4fd7-448e-8e84-27912bd8159c None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:49.002075 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49cb2b9-4fd7-448e-8e84-27912bd8159c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:49.020315 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-e49cb2b9-4fd7-448e-8e84-27912bd8159c None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:18:49.021629 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-e49cb2b9-4fd7-448e-8e84-27912bd8159c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:18:49.089613 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:49.093141 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 416/844] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:18:48 2022] POST /identity/v3/auth/tokens => generated 1163 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:18:49.094030 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49cb2b9-4fd7-448e-8e84-27912bd8159c None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:49.105570 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8a98c4-b990-4aa4-928f-7bc751d3323d None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:49.150803 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180c73dd-b7e9-4119-9d2f-174292e5df74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:49.152070 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180c73dd-b7e9-4119-9d2f-174292e5df74 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:49.152803 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180c73dd-b7e9-4119-9d2f-174292e5df74 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:49.153273 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180c73dd-b7e9-4119-9d2f-174292e5df74 None admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:49.154661 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180c73dd-b7e9-4119-9d2f-174292e5df74 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:49.155529 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180c73dd-b7e9-4119-9d2f-174292e5df74 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:49.157614 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180c73dd-b7e9-4119-9d2f-174292e5df74 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:49.181573 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:49.182128 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 429/845] 10.222.0.102 () {56 vars in 1140 bytes} [Thu Sep 1 17:18:48 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 372 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:49.189366 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180c73dd-b7e9-4119-9d2f-174292e5df74 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:49.202339 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49cb2b9-4fd7-448e-8e84-27912bd8159c None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:49.239272 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70678ce0-b870-4ee0-a523-bf5283d90240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:49.240113 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70678ce0-b870-4ee0-a523-bf5283d90240 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:49.240577 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70678ce0-b870-4ee0-a523-bf5283d90240 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:49.240994 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70678ce0-b870-4ee0-a523-bf5283d90240 None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:49.241806 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70678ce0-b870-4ee0-a523-bf5283d90240 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:49.242253 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70678ce0-b870-4ee0-a523-bf5283d90240 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:49.243765 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70678ce0-b870-4ee0-a523-bf5283d90240 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:49.249055 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 417/846] 10.222.0.102 () {56 vars in 1119 bytes} [Thu Sep 1 17:18:49 2022] GET /identity/v3/services?name=volumev3 => generated 126 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:49.254499 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70678ce0-b870-4ee0-a523-bf5283d90240 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:49.305960 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180c73dd-b7e9-4119-9d2f-174292e5df74 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:49.344865 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b97babc-5bb3-4bfa-84ae-c4ac2c88bfa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:49.345507 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b97babc-5bb3-4bfa-84ae-c4ac2c88bfa7 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:49.345822 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b97babc-5bb3-4bfa-84ae-c4ac2c88bfa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:49.347367 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b97babc-5bb3-4bfa-84ae-c4ac2c88bfa7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:49.376774 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:49.386071 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70678ce0-b870-4ee0-a523-bf5283d90240 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:49.404263 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:49.422910 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948164af-1123-498e-bf12-c7dc34922bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:49.423756 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948164af-1123-498e-bf12-c7dc34922bcc None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:49.424073 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948164af-1123-498e-bf12-c7dc34922bcc None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:49.424377 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948164af-1123-498e-bf12-c7dc34922bcc None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:49.425573 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948164af-1123-498e-bf12-c7dc34922bcc None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:49.425987 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948164af-1123-498e-bf12-c7dc34922bcc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:49.427264 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948164af-1123-498e-bf12-c7dc34922bcc None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:49.429525 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b97babc-5bb3-4bfa-84ae-c4ac2c88bfa7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:49.430960 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b97babc-5bb3-4bfa-84ae-c4ac2c88bfa7 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:49.432051 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 418/847] 10.222.0.102 () {56 vars in 1119 bytes} [Thu Sep 1 17:18:49 2022] GET /identity/v3/services?type=volumev3 => generated 126 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:18:49.435157 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948164af-1123-498e-bf12-c7dc34922bcc None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:49.569044 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 430/848] 10.222.0.102 () {58 vars in 1287 bytes} [Thu Sep 1 17:18:49 2022] GET /identity/v3/auth/tokens => generated 1163 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 17:18:49.577063 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b97babc-5bb3-4bfa-84ae-c4ac2c88bfa7 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:49.578018 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948164af-1123-498e-bf12-c7dc34922bcc None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:49.618044 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5523f9-3a86-414e-9b44-a96d8b2844e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:49.618965 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5523f9-3a86-414e-9b44-a96d8b2844e7 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:49.619393 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5523f9-3a86-414e-9b44-a96d8b2844e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:49.619845 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5523f9-3a86-414e-9b44-a96d8b2844e7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:49.631624 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5523f9-3a86-414e-9b44-a96d8b2844e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:18:49.636906 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:49.637259 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-dd5523f9-3a86-414e-9b44-a96d8b2844e7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:18:49.637259 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:49.637259 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:49.637259 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:49.637259 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:49.637259 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:49.637259 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:49.637259 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:49.637259 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:49.637259 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:49.637259 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:49.637259 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:49.637259 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:18:49.637259 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:18:49.637259 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:18:49.637259 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:18:49.638379 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:18:49.638379 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:18:49.638379 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:18:49.638379 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:18:49.638379 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:49.638379 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:49.638379 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:49.638379 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:49.638379 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:49.638379 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:49.638379 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:49.638379 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:49.638379 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:49.638379 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:49.638379 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:49.638379 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:49.639423 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:49.639423 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:49.639423 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:49.639423 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:49.639423 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:18:49.639423 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:18:49.639423 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:18:49.639423 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:18:49.639423 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:18:49.639423 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:18:49.639423 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:18:49.639423 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:18:49.642353 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 419/849] 10.222.0.102 () {56 vars in 1107 bytes} [Thu Sep 1 17:18:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:49.648834 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5523f9-3a86-414e-9b44-a96d8b2844e7 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:49.693429 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b97babc-5bb3-4bfa-84ae-c4ac2c88bfa7 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:49.708677 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:49.733815 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e0f047-8782-4d3e-9ea5-0c6dd4bbd495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:49.734509 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e0f047-8782-4d3e-9ea5-0c6dd4bbd495 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:49.734791 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e0f047-8782-4d3e-9ea5-0c6dd4bbd495 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:49.735093 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e0f047-8782-4d3e-9ea5-0c6dd4bbd495 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:49.746743 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e0f047-8782-4d3e-9ea5-0c6dd4bbd495 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:49.748224 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e0f047-8782-4d3e-9ea5-0c6dd4bbd495 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:49.764950 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-04e0f047-8782-4d3e-9ea5-0c6dd4bbd495 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:49.764950 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:49.764950 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:49.764950 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:49.764950 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:49.764950 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:49.764950 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:49.764950 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:49.764950 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:49.764950 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:49.764950 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:49.764950 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:49.764950 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:18:49.764950 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:18:49.764950 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:18:49.764950 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:18:49.766138 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:49.766138 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:49.766138 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:49.766138 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:49.766138 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:49.766138 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:49.766138 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:49.766138 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:49.766138 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:49.766138 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:49.766138 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:49.766138 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:49.766138 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:49.766138 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:49.766138 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:49.766138 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:49.766138 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:18:49.767379 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:18:49.767379 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:49.767379 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:18:49.771100 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 431/850] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:18:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:49.771991 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e0f047-8782-4d3e-9ea5-0c6dd4bbd495 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:49.786358 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5523f9-3a86-414e-9b44-a96d8b2844e7 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:49.830538 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8637af97-6fba-4389-96d5-f1fd5b82839e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:49.831308 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8637af97-6fba-4389-96d5-f1fd5b82839e None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:49.831680 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8637af97-6fba-4389-96d5-f1fd5b82839e None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:49.832145 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8637af97-6fba-4389-96d5-f1fd5b82839e None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:49.832931 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8637af97-6fba-4389-96d5-f1fd5b82839e None admin] RBAC: Adding query filter params (name=service) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:49.833368 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8637af97-6fba-4389-96d5-f1fd5b82839e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:49.834841 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8637af97-6fba-4389-96d5-f1fd5b82839e None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:49.848123 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 420/851] 10.222.0.102 () {56 vars in 1108 bytes} [Thu Sep 1 17:18:49 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:49.849204 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8637af97-6fba-4389-96d5-f1fd5b82839e None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:49.892119 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e0f047-8782-4d3e-9ea5-0c6dd4bbd495 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:49.900442 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:49.933376 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef7d601-822c-406b-8860-31f018e17c35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:49.934227 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef7d601-822c-406b-8860-31f018e17c35 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:49.934685 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef7d601-822c-406b-8860-31f018e17c35 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:49.935104 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef7d601-822c-406b-8860-31f018e17c35 None admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:49.935988 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef7d601-822c-406b-8860-31f018e17c35 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:49.936438 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef7d601-822c-406b-8860-31f018e17c35 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:49.938060 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef7d601-822c-406b-8860-31f018e17c35 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:49.959868 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 432/852] 10.222.0.102 () {56 vars in 1114 bytes} [Thu Sep 1 17:18:49 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:49.960986 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef7d601-822c-406b-8860-31f018e17c35 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:49.961756 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8637af97-6fba-4389-96d5-f1fd5b82839e None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:49.964292 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:49.999007 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f47306-86f9-4184-b87d-221de540928b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:49.999692 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f47306-86f9-4184-b87d-221de540928b None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:49.999976 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f47306-86f9-4184-b87d-221de540928b None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:50.000242 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f47306-86f9-4184-b87d-221de540928b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:50.015062 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f47306-86f9-4184-b87d-221de540928b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:50.016996 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f47306-86f9-4184-b87d-221de540928b None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:50.032622 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-d9f47306-86f9-4184-b87d-221de540928b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:50.032622 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:50.032622 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:50.032622 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:50.032622 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:50.032622 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:50.032622 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:50.032622 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:50.032622 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:50.032622 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:50.032622 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:50.032622 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:50.032622 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:18:50.032622 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:18:50.032622 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:18:50.032622 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:18:50.033622 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:50.033622 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:50.033622 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:50.033622 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:50.033622 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:50.033622 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:50.033622 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:50.033622 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:50.033622 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:50.033622 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:50.033622 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:50.033622 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:50.033622 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:50.033622 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:50.033622 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:50.033622 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:50.033622 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:18:50.034985 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:18:50.034985 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:50.034985 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:18:50.034985 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 421/853] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:18:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:50.040519 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f47306-86f9-4184-b87d-221de540928b None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:50.076429 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef7d601-822c-406b-8860-31f018e17c35 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:50.112646 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c3c013-c32f-4236-a0e2-57449c391dbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:50.113191 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c3c013-c32f-4236-a0e2-57449c391dbf None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:50.113512 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c3c013-c32f-4236-a0e2-57449c391dbf None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:50.113782 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c3c013-c32f-4236-a0e2-57449c391dbf None admin] PATH_INFO: `/v3/projects/service` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:50.120762 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c3c013-c32f-4236-a0e2-57449c391dbf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:50.122213 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c3c013-c32f-4236-a0e2-57449c391dbf None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:50.134346 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-00c3c013-c32f-4236-a0e2-57449c391dbf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:18:50.134346 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:50.134346 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:50.134346 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:50.134346 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:50.134346 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:50.134346 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:50.134346 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:50.134346 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:50.134346 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:50.134346 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:50.134346 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:50.134346 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:18:50.134346 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:18:50.134346 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:18:50.135409 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:18:50.135409 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:50.135409 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:50.135409 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:50.135409 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:50.135409 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:50.135409 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:50.135409 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:50.135409 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:50.135409 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:50.135409 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:50.135409 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:50.135409 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:50.135409 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:50.135409 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:50.135409 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:50.135409 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:50.136484 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:18:50.136484 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:18:50.136484 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:18:50.136484 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:18:50.136484 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:18:50.136484 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:18:50.136484 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:18:50.136484 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:18:50.140142 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 433/854] 10.222.0.102 () {56 vars in 1116 bytes} [Thu Sep 1 17:18:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:50.141349 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20adb9d-778e-45ee-a6d5-ac3668763b3c None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:50.141690 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20adb9d-778e-45ee-a6d5-ac3668763b3c None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:50.141964 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20adb9d-778e-45ee-a6d5-ac3668763b3c None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:50.143204 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 434/855] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:18:50 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:18:50.144529 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20adb9d-778e-45ee-a6d5-ac3668763b3c None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:50.159118 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f47306-86f9-4184-b87d-221de540928b None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:50.174580 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:50.196256 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05de4e6-001c-4794-a3b4-199e4ed091c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:50.196886 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05de4e6-001c-4794-a3b4-199e4ed091c8 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:50.197135 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05de4e6-001c-4794-a3b4-199e4ed091c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:50.197413 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05de4e6-001c-4794-a3b4-199e4ed091c8 None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:50.198048 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05de4e6-001c-4794-a3b4-199e4ed091c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:50.198345 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05de4e6-001c-4794-a3b4-199e4ed091c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:50.199797 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05de4e6-001c-4794-a3b4-199e4ed091c8 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:50.212616 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 422/856] 10.222.0.102 () {56 vars in 1114 bytes} [Thu Sep 1 17:18:50 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:18:50.215347 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56c1b12-55c0-4c92-85bc-90cf88382987 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:50.216060 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56c1b12-55c0-4c92-85bc-90cf88382987 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:50.216539 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56c1b12-55c0-4c92-85bc-90cf88382987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:50.235337 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-e56c1b12-55c0-4c92-85bc-90cf88382987 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:18:50.236531 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-e56c1b12-55c0-4c92-85bc-90cf88382987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:18:50.249096 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:50.260327 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20adb9d-778e-45ee-a6d5-ac3668763b3c None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:50.299824 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fc115e-9b1e-45e7-bf45-e220cbb6bab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:50.300448 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fc115e-9b1e-45e7-bf45-e220cbb6bab7 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:50.300695 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fc115e-9b1e-45e7-bf45-e220cbb6bab7 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:50.300965 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fc115e-9b1e-45e7-bf45-e220cbb6bab7 None admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:50.301845 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fc115e-9b1e-45e7-bf45-e220cbb6bab7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:50.301954 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fc115e-9b1e-45e7-bf45-e220cbb6bab7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:50.303718 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fc115e-9b1e-45e7-bf45-e220cbb6bab7 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:50.313351 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 423/857] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:18:50 2022] POST /identity/v3/auth/tokens => generated 1163 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:18:50.313351 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56c1b12-55c0-4c92-85bc-90cf88382987 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:50.334687 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 435/858] 10.222.0.102 () {56 vars in 1153 bytes} [Thu Sep 1 17:18:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:50.337708 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fc115e-9b1e-45e7-bf45-e220cbb6bab7 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:50.433321 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56c1b12-55c0-4c92-85bc-90cf88382987 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:50.457585 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:50.475198 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fc115e-9b1e-45e7-bf45-e220cbb6bab7 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:50.483835 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6177ba-9da2-4e74-b51d-9a7ecec87572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:50.485178 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6177ba-9da2-4e74-b51d-9a7ecec87572 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:50.485645 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6177ba-9da2-4e74-b51d-9a7ecec87572 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:50.486081 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6177ba-9da2-4e74-b51d-9a7ecec87572 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:50.496187 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6177ba-9da2-4e74-b51d-9a7ecec87572 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:50.498662 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6177ba-9da2-4e74-b51d-9a7ecec87572 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:50.511841 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346caa5f-bb81-4430-b1b4-3aa6f284012b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:50.512449 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346caa5f-bb81-4430-b1b4-3aa6f284012b None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:50.512736 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346caa5f-bb81-4430-b1b4-3aa6f284012b None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:50.513009 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346caa5f-bb81-4430-b1b4-3aa6f284012b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:50.513822 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346caa5f-bb81-4430-b1b4-3aa6f284012b None admin] RBAC: Adding query filter params (role.id=36577a6f46d642d4b81805ede9872490, scope.project.id=e78fff7f1f6e4f68b8d233ad2e5bbd4f, user.id=d55edf6cd1804006aa8269b4c6f80aec) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:50.514133 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346caa5f-bb81-4430-b1b4-3aa6f284012b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:50.515076 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-ed6177ba-9da2-4e74-b51d-9a7ecec87572 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 17:18:50.515076 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:50.515076 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:50.515076 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:50.515076 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:50.515076 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:50.515076 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:50.515076 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:50.515076 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:50.515076 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:50.515076 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:50.515076 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:50.515076 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:18:50.515076 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:18:50.515076 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:18:50.515076 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:18:50.516148 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:50.516148 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:50.516148 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:18:50.516148 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:18:50.516148 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:18:50.516148 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:18:50.516148 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:50.516148 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:50.516148 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:50.516148 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:50.516148 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:50.516148 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:50.516148 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:50.516148 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:50.516148 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:50.516148 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:50.516148 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:50.516148 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:50.517262 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:50.517262 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:50.517262 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:18:50.517262 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:18:50.517262 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:18:50.517262 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:18:50.517262 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:18:50.517262 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:18:50.517262 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 17:18:50.517262 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:18:50.517831 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346caa5f-bb81-4430-b1b4-3aa6f284012b None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:50.524035 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 424/859] 10.222.0.102 () {56 vars in 1122 bytes} [Thu Sep 1 17:18:50 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:50.525577 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6177ba-9da2-4e74-b51d-9a7ecec87572 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:50.528587 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 436/860] 10.222.0.102 () {56 vars in 1379 bytes} [Thu Sep 1 17:18:50 2022] GET /identity/v3/role_assignments?user.id=d55edf6cd1804006aa8269b4c6f80aec&scope.project.id=e78fff7f1f6e4f68b8d233ad2e5bbd4f&role.id=36577a6f46d642d4b81805ede9872490 => generated 610 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:50.535920 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:50.598291 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346caa5f-bb81-4430-b1b4-3aa6f284012b None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:50.663114 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6177ba-9da2-4e74-b51d-9a7ecec87572 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:50.706771 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba04106-f9fc-43d4-bfe3-f85736e7458d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:50.707607 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba04106-f9fc-43d4-bfe3-f85736e7458d None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:50.708092 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba04106-f9fc-43d4-bfe3-f85736e7458d None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:50.708413 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba04106-f9fc-43d4-bfe3-f85736e7458d None admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:50.709143 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba04106-f9fc-43d4-bfe3-f85736e7458d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:50.709584 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba04106-f9fc-43d4-bfe3-f85736e7458d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:50.710896 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba04106-f9fc-43d4-bfe3-f85736e7458d None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:50.720838 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346caa5f-bb81-4430-b1b4-3aa6f284012b None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:50.733627 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 425/861] 10.222.0.102 () {56 vars in 1154 bytes} [Thu Sep 1 17:18:50 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 386 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:18:50.740059 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba04106-f9fc-43d4-bfe3-f85736e7458d None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:50.781195 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489f43fc-99c5-4faa-bfca-7016de23830d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:50.782490 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489f43fc-99c5-4faa-bfca-7016de23830d None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:50.783186 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489f43fc-99c5-4faa-bfca-7016de23830d None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:50.783848 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489f43fc-99c5-4faa-bfca-7016de23830d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:50.785641 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489f43fc-99c5-4faa-bfca-7016de23830d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:50.788030 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489f43fc-99c5-4faa-bfca-7016de23830d None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:50.806770 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-489f43fc-99c5-4faa-bfca-7016de23830d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 17:18:50.806770 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:50.806770 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 17:18:50.806770 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 17:18:50.806770 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 17:18:50.806770 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 17:18:50.806770 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 17:18:50.806770 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:18:50.806770 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 17:18:50.806770 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:18:50.806770 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 17:18:50.806770 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:18:50.806770 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:50.806770 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:50.806770 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:50.806770 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:50.808951 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:50.808951 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:50.808951 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:50.808951 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:50.808951 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:50.808951 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:50.808951 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:50.808951 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 17:18:50.808951 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 17:18:50.808951 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 17:18:50.808951 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 17:18:50.808951 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:50.808951 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:50.808951 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:50.808951 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:50.808951 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:50.808951 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:50.811051 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:50.811051 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:50.811051 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:50.811051 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:50.811051 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:50.811051 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:50.811051 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:50.811051 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:50.811051 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:50.811051 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:50.811051 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 17:18:50.811051 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:18:50.811051 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 17:18:50.811051 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:18:50.818537 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 437/862] 10.222.0.102 () {56 vars in 1137 bytes} [Thu Sep 1 17:18:50 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:18:50.819690 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489f43fc-99c5-4faa-bfca-7016de23830d None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:50.841407 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:50.843554 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:50.888628 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba04106-f9fc-43d4-bfe3-f85736e7458d None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:50.928243 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8e4a1d-0205-4938-b755-d25e8c097fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:50.929111 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8e4a1d-0205-4938-b755-d25e8c097fae None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:50.929594 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8e4a1d-0205-4938-b755-d25e8c097fae None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:50.930047 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8e4a1d-0205-4938-b755-d25e8c097fae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:50.940369 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489f43fc-99c5-4faa-bfca-7016de23830d None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:50.977027 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaf1097-6f42-4530-b55a-cebbf25dcd22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:50.978075 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaf1097-6f42-4530-b55a-cebbf25dcd22 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:50.978451 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaf1097-6f42-4530-b55a-cebbf25dcd22 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:50.978864 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaf1097-6f42-4530-b55a-cebbf25dcd22 None admin] PATH_INFO: `/v3/services` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:50.979579 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaf1097-6f42-4530-b55a-cebbf25dcd22 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:50.980031 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaf1097-6f42-4530-b55a-cebbf25dcd22 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:50.981443 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaf1097-6f42-4530-b55a-cebbf25dcd22 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:50.991504 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 438/863] 10.222.0.102 () {56 vars in 1131 bytes} [Thu Sep 1 17:18:50 2022] GET /identity/v3/services?name=compute_legacy => generated 132 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:50.992756 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaf1097-6f42-4530-b55a-cebbf25dcd22 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:51.017926 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8e4a1d-0205-4938-b755-d25e8c097fae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:51.019917 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8e4a1d-0205-4938-b755-d25e8c097fae None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:51.085392 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:51.088783 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:51.109277 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaf1097-6f42-4530-b55a-cebbf25dcd22 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:51.132396 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 426/864] 10.222.0.102 () {58 vars in 1287 bytes} [Thu Sep 1 17:18:50 2022] GET /identity/v3/auth/tokens => generated 1163 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 17:18:51.138231 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8e4a1d-0205-4938-b755-d25e8c097fae None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:51.148982 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4a084e-25d7-4f09-a230-160ce1a23a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:51.149773 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4a084e-25d7-4f09-a230-160ce1a23a70 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:51.150157 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4a084e-25d7-4f09-a230-160ce1a23a70 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:51.150803 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4a084e-25d7-4f09-a230-160ce1a23a70 None admin] PATH_INFO: `/v3/services` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:51.151388 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4a084e-25d7-4f09-a230-160ce1a23a70 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:51.151797 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4a084e-25d7-4f09-a230-160ce1a23a70 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:51.153392 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4a084e-25d7-4f09-a230-160ce1a23a70 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:51.158845 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 439/865] 10.222.0.102 () {56 vars in 1131 bytes} [Thu Sep 1 17:18:50 2022] GET /identity/v3/services?type=compute_legacy => generated 132 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:51.254122 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8e4a1d-0205-4938-b755-d25e8c097fae None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:51.293498 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed6bb17-5e2c-4a96-982f-ecaab544bebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:51.294410 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed6bb17-5e2c-4a96-982f-ecaab544bebd None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:51.294837 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed6bb17-5e2c-4a96-982f-ecaab544bebd None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:51.295282 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed6bb17-5e2c-4a96-982f-ecaab544bebd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:51.306669 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed6bb17-5e2c-4a96-982f-ecaab544bebd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:51.308342 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed6bb17-5e2c-4a96-982f-ecaab544bebd None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:51.323690 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-1ed6bb17-5e2c-4a96-982f-ecaab544bebd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:51.323690 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:51.323690 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:51.323690 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:51.323690 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:51.323690 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:51.323690 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:51.323690 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:51.323690 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:51.323690 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:51.323690 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:51.323690 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:51.323690 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:18:51.323690 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:18:51.323690 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:18:51.323690 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:18:51.324860 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:51.324860 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:51.324860 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:51.324860 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:51.324860 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:51.324860 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:51.324860 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:51.324860 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:51.324860 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:51.324860 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:51.324860 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:51.324860 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:51.324860 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:51.324860 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:51.324860 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:51.324860 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:51.324860 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:18:51.325970 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:18:51.325970 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:51.325970 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:18:51.327381 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 427/866] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:18:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:51.333861 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4a084e-25d7-4f09-a230-160ce1a23a70 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:51.437285 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4a084e-25d7-4f09-a230-160ce1a23a70 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:51.463249 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec08c8f-9581-4686-a69d-6c797d978f81 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:51.463576 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec08c8f-9581-4686-a69d-6c797d978f81 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:51.463804 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec08c8f-9581-4686-a69d-6c797d978f81 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:51.466676 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 428/867] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:18:51 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Sep 01 17:18:51.473918 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a379712c-fb61-4ddd-8c70-b4e055027be0 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:51.474467 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a379712c-fb61-4ddd-8c70-b4e055027be0 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:51.474996 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a379712c-fb61-4ddd-8c70-b4e055027be0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:51.476965 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8a2727-c9d9-400c-8b1c-0482b312986e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:51.477600 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8a2727-c9d9-400c-8b1c-0482b312986e None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:51.478133 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8a2727-c9d9-400c-8b1c-0482b312986e None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:51.478409 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8a2727-c9d9-400c-8b1c-0482b312986e None admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:51.479051 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8a2727-c9d9-400c-8b1c-0482b312986e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:51.479355 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8a2727-c9d9-400c-8b1c-0482b312986e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:51.480903 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8a2727-c9d9-400c-8b1c-0482b312986e None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:51.495248 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 440/868] 10.222.0.102 () {56 vars in 1114 bytes} [Thu Sep 1 17:18:51 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:51.501887 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-a379712c-fb61-4ddd-8c70-b4e055027be0 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:18:51.502053 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8a2727-c9d9-400c-8b1c-0482b312986e None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:51.503946 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-a379712c-fb61-4ddd-8c70-b4e055027be0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:18:51.505273 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:51.533670 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:51.582140 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 429/869] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:18:51 2022] POST /identity/v3/auth/tokens => generated 1163 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 17:18:51.609783 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8a2727-c9d9-400c-8b1c-0482b312986e None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:51.645713 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1a5fe1-f6c5-4a36-8cde-d5e6d97286a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:51.646709 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1a5fe1-f6c5-4a36-8cde-d5e6d97286a5 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:51.647086 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1a5fe1-f6c5-4a36-8cde-d5e6d97286a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:51.647484 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1a5fe1-f6c5-4a36-8cde-d5e6d97286a5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:51.654790 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1a5fe1-f6c5-4a36-8cde-d5e6d97286a5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:51.656493 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1a5fe1-f6c5-4a36-8cde-d5e6d97286a5 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:51.669357 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-9b1a5fe1-f6c5-4a36-8cde-d5e6d97286a5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:18:51.669357 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:51.669357 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:51.669357 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:51.669357 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:51.669357 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:51.669357 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:51.669357 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:51.669357 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:51.669357 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:51.669357 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:51.669357 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:51.669357 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:18:51.669357 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:18:51.669357 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:18:51.670466 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:18:51.670466 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:51.670466 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:51.670466 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:51.670466 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:51.670466 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:51.670466 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:51.670466 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:51.670466 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:51.670466 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:51.670466 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:51.670466 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:51.670466 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:51.670466 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:51.670466 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:51.670466 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:51.670466 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:51.671588 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:18:51.671588 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:18:51.671588 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:18:51.671588 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:18:51.671588 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:18:51.671588 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:18:51.671588 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:18:51.671588 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:18:51.673236 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 441/870] 10.222.0.102 () {56 vars in 1116 bytes} [Thu Sep 1 17:18:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:18:51.678882 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a379712c-fb61-4ddd-8c70-b4e055027be0 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:51.788533 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a379712c-fb61-4ddd-8c70-b4e055027be0 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:51.824514 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370db574-fd7d-401b-aca6-0a683f36002d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:51.825453 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370db574-fd7d-401b-aca6-0a683f36002d None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:51.825878 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370db574-fd7d-401b-aca6-0a683f36002d None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:51.826533 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370db574-fd7d-401b-aca6-0a683f36002d None admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:51.827388 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370db574-fd7d-401b-aca6-0a683f36002d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:51.827851 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370db574-fd7d-401b-aca6-0a683f36002d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:51.829456 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370db574-fd7d-401b-aca6-0a683f36002d None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:51.842102 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 430/871] 10.222.0.102 () {56 vars in 1153 bytes} [Thu Sep 1 17:18:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:51.853363 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1a5fe1-f6c5-4a36-8cde-d5e6d97286a5 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:51.854834 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370db574-fd7d-401b-aca6-0a683f36002d None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:51.869682 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:51.919249 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:51.965074 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1a5fe1-f6c5-4a36-8cde-d5e6d97286a5 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:51.993407 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370db574-fd7d-401b-aca6-0a683f36002d None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:52.002434 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70422f3d-03f2-4769-9246-3afe2332c39e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:52.003902 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70422f3d-03f2-4769-9246-3afe2332c39e None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:52.004279 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70422f3d-03f2-4769-9246-3afe2332c39e None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:52.004646 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70422f3d-03f2-4769-9246-3afe2332c39e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:52.005495 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70422f3d-03f2-4769-9246-3afe2332c39e None admin] RBAC: Adding query filter params (role.id=f21e23654e4740158c7d4f46e726e515, scope.project.id=e78fff7f1f6e4f68b8d233ad2e5bbd4f, user.id=575a76b99ea945efb1024fbb3dc81aaf) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:52.005928 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70422f3d-03f2-4769-9246-3afe2332c39e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:52.007583 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70422f3d-03f2-4769-9246-3afe2332c39e None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:52.021347 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 442/872] 10.222.0.102 () {56 vars in 1379 bytes} [Thu Sep 1 17:18:51 2022] GET /identity/v3/role_assignments?user.id=575a76b99ea945efb1024fbb3dc81aaf&scope.project.id=e78fff7f1f6e4f68b8d233ad2e5bbd4f&role.id=f21e23654e4740158c7d4f46e726e515 => generated 610 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:52.043557 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec729e2-a6fb-4a09-846f-b822d3db9bfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:52.044159 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec729e2-a6fb-4a09-846f-b822d3db9bfb None admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:52.044436 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec729e2-a6fb-4a09-846f-b822d3db9bfb None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:52.044707 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec729e2-a6fb-4a09-846f-b822d3db9bfb None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:52.045480 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec729e2-a6fb-4a09-846f-b822d3db9bfb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:52.046906 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec729e2-a6fb-4a09-846f-b822d3db9bfb None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:52.064548 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 431/873] 10.222.0.102 () {60 vars in 1146 bytes} [Thu Sep 1 17:18:51 2022] POST /identity/v3/services => generated 252 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:18:52.606410 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6168c880-f3f9-4738-bfcd-cb764263540a None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:52.606936 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6168c880-f3f9-4738-bfcd-cb764263540a None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:52.607047 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6168c880-f3f9-4738-bfcd-cb764263540a None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:52.609283 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 443/874] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:18:52 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:18:52.614908 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6ea53e-87b8-47ea-b8cd-51d957afe1dc None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:52.615204 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6ea53e-87b8-47ea-b8cd-51d957afe1dc None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:52.615486 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6ea53e-87b8-47ea-b8cd-51d957afe1dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:52.634199 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-2b6ea53e-87b8-47ea-b8cd-51d957afe1dc None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:18:52.635403 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-2b6ea53e-87b8-47ea-b8cd-51d957afe1dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:18:52.698999 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:52.716115 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 432/875] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:18:52 2022] POST /identity/v3/auth/tokens => generated 1264 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:18:53.001084 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6168c880-f3f9-4738-bfcd-cb764263540a None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:53.111782 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6168c880-f3f9-4738-bfcd-cb764263540a None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:53.139776 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:53.148998 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0b8fcd-0ae2-4019-bde3-4400e58a7c5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:53.149657 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0b8fcd-0ae2-4019-bde3-4400e58a7c5a None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:53.149988 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0b8fcd-0ae2-4019-bde3-4400e58a7c5a None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:53.150264 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0b8fcd-0ae2-4019-bde3-4400e58a7c5a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:53.150846 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0b8fcd-0ae2-4019-bde3-4400e58a7c5a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:53.152240 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0b8fcd-0ae2-4019-bde3-4400e58a7c5a None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:53.161888 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-ac0b8fcd-0ae2-4019-bde3-4400e58a7c5a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 17:18:53.161888 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:53.161888 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 17:18:53.161888 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 17:18:53.161888 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 17:18:53.161888 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 17:18:53.161888 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 17:18:53.161888 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:18:53.161888 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 17:18:53.161888 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:18:53.161888 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 17:18:53.161888 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:18:53.161888 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:53.161888 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:53.161888 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:53.161888 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:53.161888 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:53.162893 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:53.162893 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:53.162893 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:53.162893 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:53.162893 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:53.162893 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:53.162893 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 17:18:53.162893 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 17:18:53.162893 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 17:18:53.162893 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 17:18:53.162893 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:53.162893 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:53.162893 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:53.162893 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:53.162893 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:53.162893 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:53.163736 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:53.163736 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:53.163736 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:53.163736 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:53.163736 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:53.163736 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:53.163736 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:53.163736 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:53.163736 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:53.163736 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:53.163736 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 17:18:53.163736 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:18:53.163736 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 17:18:53.163736 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:18:53.163736 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 444/876] 10.222.0.102 () {56 vars in 1131 bytes} [Thu Sep 1 17:18:52 2022] GET /identity/v3/services/object-store => generated 93 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:53.171076 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6ea53e-87b8-47ea-b8cd-51d957afe1dc None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:53.183820 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c194a1f-ae83-40f6-b342-f55cd50e9242 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:53.184107 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c194a1f-ae83-40f6-b342-f55cd50e9242 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:53.184373 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c194a1f-ae83-40f6-b342-f55cd50e9242 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:53.185955 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 445/877] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:18:53 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Sep 01 17:18:53.190896 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564205ad-0e48-4b82-b200-fa46f02b7bff None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:53.191232 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564205ad-0e48-4b82-b200-fa46f02b7bff None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:53.191510 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564205ad-0e48-4b82-b200-fa46f02b7bff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:53.212926 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-564205ad-0e48-4b82-b200-fa46f02b7bff None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:18:53.214671 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-564205ad-0e48-4b82-b200-fa46f02b7bff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:18:53.297673 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 446/878] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:18:53 2022] POST /identity/v3/auth/tokens => generated 1264 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 17:18:53.305194 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6ea53e-87b8-47ea-b8cd-51d957afe1dc None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:53.341694 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b672b34-3385-4091-8668-1b2649be52a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:53.342637 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b672b34-3385-4091-8668-1b2649be52a6 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:53.343134 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b672b34-3385-4091-8668-1b2649be52a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:53.343609 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b672b34-3385-4091-8668-1b2649be52a6 None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:53.344435 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b672b34-3385-4091-8668-1b2649be52a6 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:53.344963 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b672b34-3385-4091-8668-1b2649be52a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:53.347105 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b672b34-3385-4091-8668-1b2649be52a6 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:53.352332 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 433/879] 10.222.0.102 () {56 vars in 1127 bytes} [Thu Sep 1 17:18:53 2022] GET /identity/v3/services?name=object-store => generated 130 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:53.358411 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564205ad-0e48-4b82-b200-fa46f02b7bff None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:53.457894 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564205ad-0e48-4b82-b200-fa46f02b7bff None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:53.484219 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:53.499231 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6788c95a-08d7-4890-9ac1-acd567e331ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:53.500166 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6788c95a-08d7-4890-9ac1-acd567e331ce None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:53.500651 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6788c95a-08d7-4890-9ac1-acd567e331ce None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:53.501142 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6788c95a-08d7-4890-9ac1-acd567e331ce None admin] PATH_INFO: `/v3/services` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:53.501991 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6788c95a-08d7-4890-9ac1-acd567e331ce None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:53.502515 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6788c95a-08d7-4890-9ac1-acd567e331ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:53.504111 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6788c95a-08d7-4890-9ac1-acd567e331ce None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:53.509286 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 447/880] 10.222.0.102 () {56 vars in 1127 bytes} [Thu Sep 1 17:18:53 2022] GET /identity/v3/services?type=object-store => generated 130 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:18:53.616883 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b672b34-3385-4091-8668-1b2649be52a6 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:53.680249 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:53.728437 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b672b34-3385-4091-8668-1b2649be52a6 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:53.764833 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346c191b-3b82-4732-9d26-2f7f6a3b5395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:53.765515 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346c191b-3b82-4732-9d26-2f7f6a3b5395 None admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:53.765794 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346c191b-3b82-4732-9d26-2f7f6a3b5395 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:53.766064 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346c191b-3b82-4732-9d26-2f7f6a3b5395 None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:53.766718 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346c191b-3b82-4732-9d26-2f7f6a3b5395 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:53.768148 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346c191b-3b82-4732-9d26-2f7f6a3b5395 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:53.786944 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 434/881] 10.222.0.102 () {60 vars in 1146 bytes} [Thu Sep 1 17:18:53 2022] POST /identity/v3/services => generated 270 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:18:54.076323 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76cc3b2-4996-4af5-81a7-78550da91d08 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:54.076868 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76cc3b2-4996-4af5-81a7-78550da91d08 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:54.076963 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcf4af5-a902-4f0f-ab85-99dd4c156727 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:54.077039 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76cc3b2-4996-4af5-81a7-78550da91d08 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:54.077108 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcf4af5-a902-4f0f-ab85-99dd4c156727 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:54.077431 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcf4af5-a902-4f0f-ab85-99dd4c156727 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:54.078498 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 448/882] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:18:54 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:18:54.078881 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 435/883] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:18:54 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Sep 01 17:18:54.083905 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43af1264-78da-4118-bdc7-5e93b9953354 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:54.084190 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43af1264-78da-4118-bdc7-5e93b9953354 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:54.084462 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43af1264-78da-4118-bdc7-5e93b9953354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:54.085416 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081243fe-261d-424f-8419-381e34aa30ca None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:54.085903 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081243fe-261d-424f-8419-381e34aa30ca None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:54.086392 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081243fe-261d-424f-8419-381e34aa30ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:54.103384 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-43af1264-78da-4118-bdc7-5e93b9953354 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:18:54.104587 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-43af1264-78da-4118-bdc7-5e93b9953354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:18:54.105921 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-081243fe-261d-424f-8419-381e34aa30ca None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:18:54.107532 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-081243fe-261d-424f-8419-381e34aa30ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:18:54.184801 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 436/884] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:18:54 2022] POST /identity/v3/auth/tokens => generated 1374 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:18:54.196943 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 449/885] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:18:54 2022] POST /identity/v3/auth/tokens => generated 1374 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:18:54.458957 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43af1264-78da-4118-bdc7-5e93b9953354 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:54.465408 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081243fe-261d-424f-8419-381e34aa30ca None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:54.510004 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:54.580406 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43af1264-78da-4118-bdc7-5e93b9953354 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:54.592986 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081243fe-261d-424f-8419-381e34aa30ca None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:54.606918 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:54.624354 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8897b605-e184-499f-bece-a479ca3c5331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:54.624988 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8897b605-e184-499f-bece-a479ca3c5331 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:54.625270 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8897b605-e184-499f-bece-a479ca3c5331 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:54.625574 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8897b605-e184-499f-bece-a479ca3c5331 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:54.634509 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2529e53d-4be1-429d-97b6-257cd2ceb59d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:54.635440 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2529e53d-4be1-429d-97b6-257cd2ceb59d None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:54.635914 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2529e53d-4be1-429d-97b6-257cd2ceb59d None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:54.636399 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2529e53d-4be1-429d-97b6-257cd2ceb59d None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:54.637177 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2529e53d-4be1-429d-97b6-257cd2ceb59d None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:54.638326 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8897b605-e184-499f-bece-a479ca3c5331 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:18:54.638996 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2529e53d-4be1-429d-97b6-257cd2ceb59d None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:54.644397 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-8897b605-e184-499f-bece-a479ca3c5331 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 17:18:54.644397 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:54.644397 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:54.644397 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:54.644397 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:54.644397 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:54.644397 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:54.644397 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:54.644397 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:54.644397 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:54.644397 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:54.644397 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:54.644397 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:18:54.644397 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:18:54.644397 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:18:54.644397 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:18:54.645394 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:18:54.645394 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:18:54.645394 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:18:54.645394 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:18:54.645394 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:54.645394 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:54.645394 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:54.645394 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:54.645394 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:54.645394 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:54.645394 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:54.645394 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:54.645394 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:54.645394 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:54.645394 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:54.645394 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:54.646219 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:54.646219 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:54.646219 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:54.646219 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:54.646219 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:18:54.646219 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:18:54.646219 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:18:54.646219 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:18:54.646219 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:18:54.646219 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:18:54.646219 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 17:18:54.646219 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:18:54.650896 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 437/886] 10.222.0.102 () {56 vars in 1125 bytes} [Thu Sep 1 17:18:54 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:54.652063 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-2529e53d-4be1-429d-97b6-257cd2ceb59d None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 17:18:54.652063 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:54.652063 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 17:18:54.652063 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 17:18:54.652063 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 17:18:54.652063 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 17:18:54.652063 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 17:18:54.652063 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:18:54.652063 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 17:18:54.652063 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:18:54.652063 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 17:18:54.652063 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:18:54.652063 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:54.652063 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:54.652063 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:54.652063 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:54.652063 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:54.652988 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:54.652988 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:54.652988 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:54.652988 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:54.652988 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:54.652988 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:54.652988 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 17:18:54.652988 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 17:18:54.652988 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 17:18:54.652988 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 17:18:54.652988 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:54.652988 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:54.652988 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:54.652988 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:54.652988 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:54.652988 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:54.654327 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:54.654327 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:54.654327 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:54.654327 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:54.654327 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:54.654327 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:54.654327 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:54.654327 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:54.654327 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:54.654327 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:54.654327 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 17:18:54.654327 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:18:54.654327 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 17:18:54.654327 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:18:54.655075 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8897b605-e184-499f-bece-a479ca3c5331 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:54.663441 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 450/887] 10.222.0.102 () {56 vars in 1119 bytes} [Thu Sep 1 17:18:54 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:18:54.664631 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2529e53d-4be1-429d-97b6-257cd2ceb59d None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:54.769131 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2529e53d-4be1-429d-97b6-257cd2ceb59d None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:54.791155 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8897b605-e184-499f-bece-a479ca3c5331 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:54.793068 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:54.815867 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a59f734-bc36-4058-9493-63efcfa6a5b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:54.816770 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a59f734-bc36-4058-9493-63efcfa6a5b3 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:54.817243 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a59f734-bc36-4058-9493-63efcfa6a5b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:54.817741 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a59f734-bc36-4058-9493-63efcfa6a5b3 None admin] PATH_INFO: `/v3/services` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:54.818603 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a59f734-bc36-4058-9493-63efcfa6a5b3 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:54.819145 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a59f734-bc36-4058-9493-63efcfa6a5b3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:54.820717 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a59f734-bc36-4058-9493-63efcfa6a5b3 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:54.831285 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 451/888] 10.222.0.102 () {56 vars in 1119 bytes} [Thu Sep 1 17:18:54 2022] GET /identity/v3/services?name=volumev3 => generated 126 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:54.832434 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a59f734-bc36-4058-9493-63efcfa6a5b3 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:54.838409 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2da92f-b08c-462a-a995-865189574cb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:54.839074 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2da92f-b08c-462a-a995-865189574cb9 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:54.839359 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2da92f-b08c-462a-a995-865189574cb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:54.839633 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2da92f-b08c-462a-a995-865189574cb9 None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:54.840299 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2da92f-b08c-462a-a995-865189574cb9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:54.840602 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2da92f-b08c-462a-a995-865189574cb9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:54.842016 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2da92f-b08c-462a-a995-865189574cb9 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:54.856498 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 438/889] 10.222.0.102 () {56 vars in 1120 bytes} [Thu Sep 1 17:18:54 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 340 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:54.863201 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2da92f-b08c-462a-a995-865189574cb9 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:54.886812 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:54.954201 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a59f734-bc36-4058-9493-63efcfa6a5b3 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:54.994171 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2da92f-b08c-462a-a995-865189574cb9 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:55.003765 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f3e9c8-870c-4b11-b9e9-67229c1ccde5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:55.004386 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f3e9c8-870c-4b11-b9e9-67229c1ccde5 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:55.004669 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f3e9c8-870c-4b11-b9e9-67229c1ccde5 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:55.004938 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f3e9c8-870c-4b11-b9e9-67229c1ccde5 None admin] PATH_INFO: `/v3/services` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:55.005779 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f3e9c8-870c-4b11-b9e9-67229c1ccde5 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:55.006085 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f3e9c8-870c-4b11-b9e9-67229c1ccde5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:55.008060 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f3e9c8-870c-4b11-b9e9-67229c1ccde5 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:55.015545 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 452/890] 10.222.0.102 () {56 vars in 1119 bytes} [Thu Sep 1 17:18:54 2022] GET /identity/v3/services?type=volumev3 => generated 364 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:55.021276 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f3e9c8-870c-4b11-b9e9-67229c1ccde5 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:55.039164 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aee64aa-b213-4faa-92ae-57f0674412f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:55.040159 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aee64aa-b213-4faa-92ae-57f0674412f8 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:55.040634 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aee64aa-b213-4faa-92ae-57f0674412f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:55.041121 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aee64aa-b213-4faa-92ae-57f0674412f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:55.053943 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aee64aa-b213-4faa-92ae-57f0674412f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:55.055800 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aee64aa-b213-4faa-92ae-57f0674412f8 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:55.072369 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-0aee64aa-b213-4faa-92ae-57f0674412f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:55.072369 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:55.072369 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:55.072369 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:55.072369 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:55.072369 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:55.072369 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:55.072369 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:55.072369 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:55.072369 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:55.072369 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:55.072369 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:55.072369 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:18:55.072369 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:18:55.072369 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:18:55.072369 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:18:55.073684 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:55.073684 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:55.073684 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:55.073684 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:55.073684 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:55.073684 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:55.073684 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:55.073684 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:55.073684 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:55.073684 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:55.073684 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:55.073684 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:55.073684 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:55.073684 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:55.073684 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:55.073684 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:55.073684 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:18:55.074557 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:18:55.074557 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:55.074557 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:18:55.076589 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 439/891] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:18:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:55.082869 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aee64aa-b213-4faa-92ae-57f0674412f8 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:55.106191 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:55.147511 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f3e9c8-870c-4b11-b9e9-67229c1ccde5 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:55.150000 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:55.194815 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c7bb70-b1d6-464f-a9b9-bb07cb60b914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:55.196000 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c7bb70-b1d6-464f-a9b9-bb07cb60b914 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:55.196495 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c7bb70-b1d6-464f-a9b9-bb07cb60b914 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:55.196962 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c7bb70-b1d6-464f-a9b9-bb07cb60b914 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:55.198465 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c7bb70-b1d6-464f-a9b9-bb07cb60b914 None admin] RBAC: Adding query filter params (interface=public, service_id=fee85642f27c4eab86afb232325b49c0, region_id=RegionOne) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:55.199272 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c7bb70-b1d6-464f-a9b9-bb07cb60b914 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:55.201327 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c7bb70-b1d6-464f-a9b9-bb07cb60b914 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:55.213008 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 453/892] 10.222.0.102 () {56 vars in 1256 bytes} [Thu Sep 1 17:18:55 2022] GET /identity/v3/endpoints?service_id=fee85642f27c4eab86afb232325b49c0&interface=public®ion_id=RegionOne => generated 195 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:18:55.215312 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aee64aa-b213-4faa-92ae-57f0674412f8 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:55.217663 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c7bb70-b1d6-464f-a9b9-bb07cb60b914 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:55.255095 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80393318-48a2-4fdd-a081-cb12df2173a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:55.256185 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80393318-48a2-4fdd-a081-cb12df2173a9 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:55.256661 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80393318-48a2-4fdd-a081-cb12df2173a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:55.257125 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80393318-48a2-4fdd-a081-cb12df2173a9 None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:55.257990 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80393318-48a2-4fdd-a081-cb12df2173a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:55.258504 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80393318-48a2-4fdd-a081-cb12df2173a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:55.260080 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80393318-48a2-4fdd-a081-cb12df2173a9 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:55.274422 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 440/893] 10.222.0.102 () {56 vars in 1114 bytes} [Thu Sep 1 17:18:55 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:55.281857 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80393318-48a2-4fdd-a081-cb12df2173a9 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:55.333798 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c7bb70-b1d6-464f-a9b9-bb07cb60b914 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:55.371752 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377d12cd-f121-4bee-a527-09c20a775bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:55.372264 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377d12cd-f121-4bee-a527-09c20a775bca None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:55.372546 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377d12cd-f121-4bee-a527-09c20a775bca None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:55.372823 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377d12cd-f121-4bee-a527-09c20a775bca None admin] PATH_INFO: `/v3/services` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:55.373471 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377d12cd-f121-4bee-a527-09c20a775bca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:55.374840 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377d12cd-f121-4bee-a527-09c20a775bca None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:55.386536 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 454/894] 10.222.0.102 () {56 vars in 1092 bytes} [Thu Sep 1 17:18:55 2022] GET /identity/v3/services => generated 1274 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:18:55.397343 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:55.420247 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80393318-48a2-4fdd-a081-cb12df2173a9 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:55.461285 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c815d17-094a-4c04-9b56-12246885e399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:55.461959 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c815d17-094a-4c04-9b56-12246885e399 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:55.462238 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c815d17-094a-4c04-9b56-12246885e399 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:55.462512 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c815d17-094a-4c04-9b56-12246885e399 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:55.471118 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c815d17-094a-4c04-9b56-12246885e399 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:55.472634 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c815d17-094a-4c04-9b56-12246885e399 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:55.486515 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-5c815d17-094a-4c04-9b56-12246885e399 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 17:18:55.486515 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:55.486515 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:55.486515 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:55.486515 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:55.486515 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:55.486515 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:55.486515 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:55.486515 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:55.486515 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:55.486515 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:55.486515 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:55.486515 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:18:55.486515 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:18:55.486515 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:18:55.486515 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:18:55.487573 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:55.487573 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:55.487573 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:18:55.487573 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:18:55.487573 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:18:55.487573 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:18:55.487573 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:55.487573 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:55.487573 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:55.487573 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:55.487573 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:55.487573 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:55.487573 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:55.487573 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:55.487573 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:55.487573 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:55.487573 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:55.487573 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:55.488479 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:55.488479 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:55.488479 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:18:55.488479 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:18:55.488479 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:18:55.488479 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:18:55.488479 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:18:55.488479 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:18:55.488479 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 17:18:55.488479 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:18:55.492603 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 441/895] 10.222.0.102 () {56 vars in 1122 bytes} [Thu Sep 1 17:18:55 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:18:55.494352 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377d12cd-f121-4bee-a527-09c20a775bca None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:55.529979 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:55.609372 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377d12cd-f121-4bee-a527-09c20a775bca None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:55.645544 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cbe876-86df-4de7-b34e-516be7601def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:55.646423 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cbe876-86df-4de7-b34e-516be7601def None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:55.646844 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cbe876-86df-4de7-b34e-516be7601def None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:55.647255 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cbe876-86df-4de7-b34e-516be7601def None admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:55.648212 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cbe876-86df-4de7-b34e-516be7601def None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:55.648516 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cbe876-86df-4de7-b34e-516be7601def None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:55.649220 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb71bdf6-b119-4ffe-82c3-fc3b5c41d4ab None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:55.649545 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb71bdf6-b119-4ffe-82c3-fc3b5c41d4ab None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:55.649823 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb71bdf6-b119-4ffe-82c3-fc3b5c41d4ab None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:55.649981 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cbe876-86df-4de7-b34e-516be7601def None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:55.651298 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 442/896] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:18:55 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:18:55.656487 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f843919-6e51-4cc6-b882-72a6f0073c71 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:55.657027 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f843919-6e51-4cc6-b882-72a6f0073c71 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:55.657551 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f843919-6e51-4cc6-b882-72a6f0073c71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:55.673561 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 455/897] 10.222.0.102 () {56 vars in 1154 bytes} [Thu Sep 1 17:18:55 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 386 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:55.682267 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cbe876-86df-4de7-b34e-516be7601def None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:55.683840 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-1f843919-6e51-4cc6-b882-72a6f0073c71 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:18:55.685246 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-1f843919-6e51-4cc6-b882-72a6f0073c71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:18:55.768895 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 443/898] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:18:55 2022] POST /identity/v3/auth/tokens => generated 1374 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:18:55.783111 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30afdb23-bc21-4785-9eac-526ec8515b57 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:55.783451 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30afdb23-bc21-4785-9eac-526ec8515b57 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:55.783888 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30afdb23-bc21-4785-9eac-526ec8515b57 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:55.786164 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 444/899] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:18:55 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Sep 01 17:18:55.791434 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba14823b-3c11-4488-9817-d698243f1992 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:55.791995 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba14823b-3c11-4488-9817-d698243f1992 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:55.792150 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba14823b-3c11-4488-9817-d698243f1992 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:55.802773 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:55.816671 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cbe876-86df-4de7-b34e-516be7601def None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:55.817000 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-ba14823b-3c11-4488-9817-d698243f1992 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:18:55.818591 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-ba14823b-3c11-4488-9817-d698243f1992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:18:55.858555 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a5b2f9-1428-4d5b-b661-7bddac495e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:55.859156 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a5b2f9-1428-4d5b-b661-7bddac495e6f None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:55.859431 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a5b2f9-1428-4d5b-b661-7bddac495e6f None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:55.859700 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a5b2f9-1428-4d5b-b661-7bddac495e6f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:55.893019 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:55.903417 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 445/900] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:18:55 2022] POST /identity/v3/auth/tokens => generated 1374 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 17:18:55.949717 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a5b2f9-1428-4d5b-b661-7bddac495e6f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:55.951390 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a5b2f9-1428-4d5b-b661-7bddac495e6f None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:56.050340 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba14823b-3c11-4488-9817-d698243f1992 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:56.061367 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:56.081412 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 456/901] 10.222.0.102 () {58 vars in 1287 bytes} [Thu Sep 1 17:18:55 2022] GET /identity/v3/auth/tokens => generated 1374 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 01 17:18:56.084425 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a5b2f9-1428-4d5b-b661-7bddac495e6f None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:56.172785 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba14823b-3c11-4488-9817-d698243f1992 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:56.201038 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a5b2f9-1428-4d5b-b661-7bddac495e6f None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:56.214644 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebd3387-4534-4b24-8cbf-9f81a0816bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:56.215257 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebd3387-4534-4b24-8cbf-9f81a0816bef None admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:56.215547 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebd3387-4534-4b24-8cbf-9f81a0816bef None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:56.216111 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebd3387-4534-4b24-8cbf-9f81a0816bef None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:56.216461 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebd3387-4534-4b24-8cbf-9f81a0816bef None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:56.217876 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebd3387-4534-4b24-8cbf-9f81a0816bef None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:56.234011 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 446/902] 10.222.0.102 () {60 vars in 1146 bytes} [Thu Sep 1 17:18:56 2022] POST /identity/v3/services => generated 242 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:18:56.235068 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebd3387-4534-4b24-8cbf-9f81a0816bef None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:56.250346 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07225d24-e2bf-40c5-b445-7a9eabe24934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:56.252220 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07225d24-e2bf-40c5-b445-7a9eabe24934 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:56.253217 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07225d24-e2bf-40c5-b445-7a9eabe24934 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:56.253755 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07225d24-e2bf-40c5-b445-7a9eabe24934 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:56.271443 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07225d24-e2bf-40c5-b445-7a9eabe24934 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:56.274026 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07225d24-e2bf-40c5-b445-7a9eabe24934 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:56.297477 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-07225d24-e2bf-40c5-b445-7a9eabe24934 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:56.297477 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:56.297477 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:56.297477 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:56.297477 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:56.297477 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:56.297477 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:56.297477 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:56.297477 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:56.297477 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:56.297477 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:56.297477 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:56.297477 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:18:56.297477 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:18:56.297477 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:18:56.297477 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:18:56.298541 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:56.298541 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:56.298541 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:56.298541 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:56.298541 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:56.298541 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:56.298541 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:56.298541 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:56.298541 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:56.298541 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:56.298541 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:56.298541 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:56.298541 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:56.298541 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:56.298541 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:56.298541 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:56.298541 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:18:56.300006 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:18:56.300006 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:18:56.300006 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:18:56.302324 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 457/903] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:18:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:18:56.309351 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07225d24-e2bf-40c5-b445-7a9eabe24934 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:56.314924 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:56.359497 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:56.370881 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebd3387-4534-4b24-8cbf-9f81a0816bef None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:56.417776 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cab444-92f1-4ae4-aaf5-77c0dafdf895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:56.418694 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cab444-92f1-4ae4-aaf5-77c0dafdf895 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:56.419141 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cab444-92f1-4ae4-aaf5-77c0dafdf895 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:56.419627 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cab444-92f1-4ae4-aaf5-77c0dafdf895 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:56.421024 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cab444-92f1-4ae4-aaf5-77c0dafdf895 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:56.422800 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cab444-92f1-4ae4-aaf5-77c0dafdf895 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:56.433676 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-60cab444-92f1-4ae4-aaf5-77c0dafdf895 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 17:18:56.433676 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:56.433676 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 17:18:56.433676 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 17:18:56.433676 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 17:18:56.433676 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 17:18:56.433676 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 17:18:56.433676 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:18:56.433676 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 17:18:56.433676 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:18:56.433676 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 17:18:56.433676 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:18:56.433676 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:56.433676 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:56.433676 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:56.433676 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:56.434809 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:56.434809 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:56.434809 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:56.434809 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:56.434809 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:56.434809 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:56.434809 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:56.434809 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 17:18:56.434809 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 17:18:56.434809 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 17:18:56.434809 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 17:18:56.434809 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:56.434809 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:56.434809 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:56.434809 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:56.434809 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:56.434809 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:56.435832 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:56.435832 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:56.435832 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:56.435832 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:56.435832 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:56.435832 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:56.435832 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:56.435832 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:56.435832 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:56.435832 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:56.435832 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 17:18:56.435832 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:18:56.435832 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 17:18:56.435832 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:18:56.439714 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 447/904] 10.222.0.102 () {56 vars in 1137 bytes} [Thu Sep 1 17:18:56 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:56.441486 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cab444-92f1-4ae4-aaf5-77c0dafdf895 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:56.446570 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07225d24-e2bf-40c5-b445-7a9eabe24934 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:56.489468 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c530a60-3f29-4635-ad56-f8cb8d5e2af0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:56.490478 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c530a60-3f29-4635-ad56-f8cb8d5e2af0 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:56.490757 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c530a60-3f29-4635-ad56-f8cb8d5e2af0 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:56.491022 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c530a60-3f29-4635-ad56-f8cb8d5e2af0 None admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:56.492009 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c530a60-3f29-4635-ad56-f8cb8d5e2af0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:56.492432 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c530a60-3f29-4635-ad56-f8cb8d5e2af0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:56.493975 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c530a60-3f29-4635-ad56-f8cb8d5e2af0 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:56.507379 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 458/905] 10.222.0.102 () {56 vars in 1114 bytes} [Thu Sep 1 17:18:56 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:56.512997 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c530a60-3f29-4635-ad56-f8cb8d5e2af0 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:56.554278 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cab444-92f1-4ae4-aaf5-77c0dafdf895 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:56.577239 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:56.593856 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503f5d54-97bc-4764-99cc-8e1d70b65af4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:56.594467 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503f5d54-97bc-4764-99cc-8e1d70b65af4 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:56.594742 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503f5d54-97bc-4764-99cc-8e1d70b65af4 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:56.595016 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503f5d54-97bc-4764-99cc-8e1d70b65af4 None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:56.595645 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503f5d54-97bc-4764-99cc-8e1d70b65af4 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:56.595944 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503f5d54-97bc-4764-99cc-8e1d70b65af4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:56.597914 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503f5d54-97bc-4764-99cc-8e1d70b65af4 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:56.603712 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 448/906] 10.222.0.102 () {56 vars in 1131 bytes} [Thu Sep 1 17:18:56 2022] GET /identity/v3/services?name=compute_legacy => generated 132 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:18:56.610597 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503f5d54-97bc-4764-99cc-8e1d70b65af4 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:56.641642 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:56.651671 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c530a60-3f29-4635-ad56-f8cb8d5e2af0 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:56.696021 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665beb1c-44db-43ce-9c2a-9945fc17616f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:56.696893 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665beb1c-44db-43ce-9c2a-9945fc17616f None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:56.697625 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665beb1c-44db-43ce-9c2a-9945fc17616f None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:56.698007 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665beb1c-44db-43ce-9c2a-9945fc17616f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:56.709145 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665beb1c-44db-43ce-9c2a-9945fc17616f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:56.710706 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665beb1c-44db-43ce-9c2a-9945fc17616f None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:56.722826 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-665beb1c-44db-43ce-9c2a-9945fc17616f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:18:56.722826 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:56.722826 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:56.722826 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:56.722826 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:56.722826 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:56.722826 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:56.722826 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:56.722826 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:56.722826 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:56.722826 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:56.722826 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:56.722826 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:18:56.722826 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:18:56.722826 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:18:56.723892 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:18:56.723892 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:56.723892 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:56.723892 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:56.723892 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:56.723892 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:56.723892 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:56.723892 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:56.723892 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:56.723892 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:56.723892 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:56.723892 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:56.723892 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:56.723892 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:56.723892 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:56.723892 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:56.723892 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:56.724942 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:18:56.724942 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:18:56.724942 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:18:56.724942 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:18:56.724942 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:18:56.724942 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:18:56.724942 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:18:56.724942 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:18:56.731411 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 459/907] 10.222.0.102 () {56 vars in 1116 bytes} [Thu Sep 1 17:18:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:56.732702 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665beb1c-44db-43ce-9c2a-9945fc17616f None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:56.739010 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503f5d54-97bc-4764-99cc-8e1d70b65af4 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:56.776114 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48cdfd5-0190-459b-a606-0e124e6e1876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:56.776922 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48cdfd5-0190-459b-a606-0e124e6e1876 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:56.777293 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48cdfd5-0190-459b-a606-0e124e6e1876 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:56.777708 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48cdfd5-0190-459b-a606-0e124e6e1876 None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:56.778528 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48cdfd5-0190-459b-a606-0e124e6e1876 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:56.778925 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48cdfd5-0190-459b-a606-0e124e6e1876 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:56.780492 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48cdfd5-0190-459b-a606-0e124e6e1876 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:56.791971 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 449/908] 10.222.0.102 () {56 vars in 1131 bytes} [Thu Sep 1 17:18:56 2022] GET /identity/v3/services?type=compute_legacy => generated 388 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:18:56.793181 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48cdfd5-0190-459b-a606-0e124e6e1876 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:56.855279 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:56.872155 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665beb1c-44db-43ce-9c2a-9945fc17616f None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:56.909345 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0edfac6-88a8-44ae-a1f3-22ba492e6bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:56.909927 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0edfac6-88a8-44ae-a1f3-22ba492e6bde None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:56.910300 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0edfac6-88a8-44ae-a1f3-22ba492e6bde None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:56.910666 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0edfac6-88a8-44ae-a1f3-22ba492e6bde None admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:56.911444 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0edfac6-88a8-44ae-a1f3-22ba492e6bde None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:56.911845 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0edfac6-88a8-44ae-a1f3-22ba492e6bde None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:56.913640 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0edfac6-88a8-44ae-a1f3-22ba492e6bde None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:56.919729 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48cdfd5-0190-459b-a606-0e124e6e1876 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:56.932044 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 460/909] 10.222.0.102 () {56 vars in 1153 bytes} [Thu Sep 1 17:18:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:56.934139 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0edfac6-88a8-44ae-a1f3-22ba492e6bde None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:56.957038 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:56.964734 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c33bc7-9f2e-4916-b8b1-4dee845870f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:56.965534 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c33bc7-9f2e-4916-b8b1-4dee845870f4 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:56.966148 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c33bc7-9f2e-4916-b8b1-4dee845870f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:56.966368 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c33bc7-9f2e-4916-b8b1-4dee845870f4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:56.967214 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c33bc7-9f2e-4916-b8b1-4dee845870f4 None admin] RBAC: Adding query filter params (interface=public, service_id=cda7ae73f15d4fefae879a7ceafda004, region_id=RegionOne) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:56.967717 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c33bc7-9f2e-4916-b8b1-4dee845870f4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:56.969344 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c33bc7-9f2e-4916-b8b1-4dee845870f4 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:56.980656 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 450/910] 10.222.0.102 () {56 vars in 1256 bytes} [Thu Sep 1 17:18:56 2022] GET /identity/v3/endpoints?service_id=cda7ae73f15d4fefae879a7ceafda004&interface=public®ion_id=RegionOne => generated 195 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:56.981725 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c33bc7-9f2e-4916-b8b1-4dee845870f4 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:57.058959 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0edfac6-88a8-44ae-a1f3-22ba492e6bde None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:57.101421 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddfcb01-f597-4fa2-a763-613e127f7d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:57.102072 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddfcb01-f597-4fa2-a763-613e127f7d3a None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:57.102470 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddfcb01-f597-4fa2-a763-613e127f7d3a None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:57.103090 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddfcb01-f597-4fa2-a763-613e127f7d3a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:57.103919 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddfcb01-f597-4fa2-a763-613e127f7d3a None admin] RBAC: Adding query filter params (role.id=9329b00eca174ca5a29094e84e243fa4, scope.project.id=e78fff7f1f6e4f68b8d233ad2e5bbd4f, user.id=575a76b99ea945efb1024fbb3dc81aaf) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:57.104324 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddfcb01-f597-4fa2-a763-613e127f7d3a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:57.105522 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c33bc7-9f2e-4916-b8b1-4dee845870f4 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:57.106228 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddfcb01-f597-4fa2-a763-613e127f7d3a None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:57.113344 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 461/911] 10.222.0.102 () {56 vars in 1379 bytes} [Thu Sep 1 17:18:56 2022] GET /identity/v3/role_assignments?user.id=575a76b99ea945efb1024fbb3dc81aaf&scope.project.id=e78fff7f1f6e4f68b8d233ad2e5bbd4f&role.id=9329b00eca174ca5a29094e84e243fa4 => generated 260 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:57.126736 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:57.153419 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1355aa7a-8ee4-4f08-86f0-750204b6d5e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:57.154443 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1355aa7a-8ee4-4f08-86f0-750204b6d5e7 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:57.155006 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1355aa7a-8ee4-4f08-86f0-750204b6d5e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:57.155472 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1355aa7a-8ee4-4f08-86f0-750204b6d5e7 None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:57.156328 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1355aa7a-8ee4-4f08-86f0-750204b6d5e7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:57.157951 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1355aa7a-8ee4-4f08-86f0-750204b6d5e7 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:57.165444 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 451/912] 10.222.0.102 () {56 vars in 1092 bytes} [Thu Sep 1 17:18:56 2022] GET /identity/v3/services => generated 1504 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:18:57.474833 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea135ad-0b70-4cd4-934e-be4af9be0016 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:57.475333 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea135ad-0b70-4cd4-934e-be4af9be0016 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:57.475333 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea135ad-0b70-4cd4-934e-be4af9be0016 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:57.476876 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 462/913] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:18:57 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:18:57.482219 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1163d70-ae3e-46a6-8fe6-c63b6071f40f None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:57.482500 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1163d70-ae3e-46a6-8fe6-c63b6071f40f None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:57.482772 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1163d70-ae3e-46a6-8fe6-c63b6071f40f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:57.501774 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-c1163d70-ae3e-46a6-8fe6-c63b6071f40f None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:18:57.503003 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-c1163d70-ae3e-46a6-8fe6-c63b6071f40f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:18:57.571844 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 452/914] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:18:57 2022] POST /identity/v3/auth/tokens => generated 1476 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:18:57.847753 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea135ad-0b70-4cd4-934e-be4af9be0016 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:57.931022 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:57.951203 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea135ad-0b70-4cd4-934e-be4af9be0016 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:57.988561 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54da28c8-fc64-4817-b4f2-a78cd717a4c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:57.989206 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54da28c8-fc64-4817-b4f2-a78cd717a4c1 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:57.989498 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54da28c8-fc64-4817-b4f2-a78cd717a4c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:57.989768 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54da28c8-fc64-4817-b4f2-a78cd717a4c1 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:57.990346 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54da28c8-fc64-4817-b4f2-a78cd717a4c1 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:57.991712 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54da28c8-fc64-4817-b4f2-a78cd717a4c1 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:58.000624 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-54da28c8-fc64-4817-b4f2-a78cd717a4c1 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 17:18:58.000624 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:58.000624 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 17:18:58.000624 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 17:18:58.000624 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 17:18:58.000624 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 17:18:58.000624 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 17:18:58.000624 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:18:58.000624 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 17:18:58.000624 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:18:58.000624 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 17:18:58.000624 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:18:58.000624 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:58.000624 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:58.000624 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:58.000624 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:58.000624 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:58.001597 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:58.001597 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:58.001597 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:58.001597 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:58.001597 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:58.001597 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:58.001597 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 17:18:58.001597 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 17:18:58.001597 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 17:18:58.001597 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 17:18:58.001597 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:58.001597 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:58.001597 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:58.001597 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:58.001597 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:58.001597 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:58.002511 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:58.002511 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:58.002511 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:58.002511 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:58.002511 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:58.002511 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:58.002511 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:58.002511 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:58.002511 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:58.002511 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:58.002511 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 17:18:58.002511 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:18:58.002511 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 17:18:58.002511 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:18:58.002511 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 463/915] 10.222.0.102 () {56 vars in 1119 bytes} [Thu Sep 1 17:18:57 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:58.008194 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1163d70-ae3e-46a6-8fe6-c63b6071f40f None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:58.121101 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1163d70-ae3e-46a6-8fe6-c63b6071f40f None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:58.129543 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:58.157890 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a2492b-39f1-43f3-a8f0-381b39d17963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:58.158568 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a2492b-39f1-43f3-a8f0-381b39d17963 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:58.159433 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a2492b-39f1-43f3-a8f0-381b39d17963 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:58.159707 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a2492b-39f1-43f3-a8f0-381b39d17963 None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:58.160341 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a2492b-39f1-43f3-a8f0-381b39d17963 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:58.160648 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a2492b-39f1-43f3-a8f0-381b39d17963 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:58.162051 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a2492b-39f1-43f3-a8f0-381b39d17963 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:58.167643 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 453/916] 10.222.0.102 () {56 vars in 1119 bytes} [Thu Sep 1 17:18:58 2022] GET /identity/v3/services?name=volumev3 => generated 126 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:58.175356 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54da28c8-fc64-4817-b4f2-a78cd717a4c1 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:58.279618 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5436a056-1b4d-4226-96ac-a49af33b0765 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:58.280493 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5436a056-1b4d-4226-96ac-a49af33b0765 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:58.280493 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5436a056-1b4d-4226-96ac-a49af33b0765 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:58.282779 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 454/917] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:18:58 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:18:58.285201 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54da28c8-fc64-4817-b4f2-a78cd717a4c1 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:58.287411 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8049c5bb-2076-471d-bf4b-12f7dd692eba None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:58.287768 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8049c5bb-2076-471d-bf4b-12f7dd692eba None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:58.288122 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8049c5bb-2076-471d-bf4b-12f7dd692eba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:58.309231 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-8049c5bb-2076-471d-bf4b-12f7dd692eba None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:18:58.311092 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-8049c5bb-2076-471d-bf4b-12f7dd692eba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:18:58.327273 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761390e0-0b34-4486-91b3-74a7fbb8f5d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:58.328247 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761390e0-0b34-4486-91b3-74a7fbb8f5d9 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:58.328729 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761390e0-0b34-4486-91b3-74a7fbb8f5d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:58.329176 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761390e0-0b34-4486-91b3-74a7fbb8f5d9 None admin] PATH_INFO: `/v3/services` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:58.330032 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761390e0-0b34-4486-91b3-74a7fbb8f5d9 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:58.331512 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761390e0-0b34-4486-91b3-74a7fbb8f5d9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:58.333366 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761390e0-0b34-4486-91b3-74a7fbb8f5d9 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:58.340607 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 464/918] 10.222.0.102 () {56 vars in 1119 bytes} [Thu Sep 1 17:18:58 2022] GET /identity/v3/services?type=volumev3 => generated 364 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:58.347263 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761390e0-0b34-4486-91b3-74a7fbb8f5d9 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:58.369677 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:58.401424 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 455/919] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:18:58 2022] POST /identity/v3/auth/tokens => generated 1476 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 17:18:58.469062 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761390e0-0b34-4486-91b3-74a7fbb8f5d9 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:58.504563 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0148b316-6781-41db-a842-e78f95e565a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:58.505846 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0148b316-6781-41db-a842-e78f95e565a3 None admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:58.506276 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0148b316-6781-41db-a842-e78f95e565a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:58.506731 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0148b316-6781-41db-a842-e78f95e565a3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:58.507550 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0148b316-6781-41db-a842-e78f95e565a3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:58.509163 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0148b316-6781-41db-a842-e78f95e565a3 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:58.531661 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 465/920] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:18:58 2022] POST /identity/v3/endpoints => generated 348 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:18:58.673863 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8049c5bb-2076-471d-bf4b-12f7dd692eba None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:58.778060 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:58.781781 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8049c5bb-2076-471d-bf4b-12f7dd692eba None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:58.825779 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba8717b-434a-4939-804e-5be1b17bf2b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:58.826568 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba8717b-434a-4939-804e-5be1b17bf2b0 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:58.826848 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba8717b-434a-4939-804e-5be1b17bf2b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:58.827118 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba8717b-434a-4939-804e-5be1b17bf2b0 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:58.827705 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba8717b-434a-4939-804e-5be1b17bf2b0 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:58.829112 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba8717b-434a-4939-804e-5be1b17bf2b0 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:58.838405 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-fba8717b-434a-4939-804e-5be1b17bf2b0 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 17:18:58.838405 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:58.838405 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 17:18:58.838405 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 17:18:58.838405 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 17:18:58.838405 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 17:18:58.838405 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 17:18:58.838405 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:18:58.838405 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 17:18:58.838405 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:18:58.838405 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 17:18:58.838405 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:18:58.838405 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:58.838405 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:58.838405 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:58.838405 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:58.838405 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:58.839411 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:58.839411 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:58.839411 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:58.839411 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:58.839411 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:58.839411 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:58.839411 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 17:18:58.839411 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 17:18:58.839411 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 17:18:58.839411 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 17:18:58.839411 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:58.839411 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:58.839411 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:58.839411 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:58.839411 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:58.839411 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:58.840638 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:58.840638 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:58.840638 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:58.840638 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:58.840638 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:58.840638 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:58.840638 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:58.840638 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:58.840638 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:58.840638 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:58.840638 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 17:18:58.840638 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:18:58.840638 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 17:18:58.840638 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:18:58.840638 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 456/921] 10.222.0.102 () {56 vars in 1131 bytes} [Thu Sep 1 17:18:58 2022] GET /identity/v3/services/object-store => generated 93 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:58.845685 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0148b316-6781-41db-a842-e78f95e565a3 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:58.913971 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:58.960665 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0148b316-6781-41db-a842-e78f95e565a3 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:58.997680 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae8bcc8-f205-4fcc-842e-beaf0c94a85e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:58.998291 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae8bcc8-f205-4fcc-842e-beaf0c94a85e None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:58.998574 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae8bcc8-f205-4fcc-842e-beaf0c94a85e None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:58.998846 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae8bcc8-f205-4fcc-842e-beaf0c94a85e None admin] PATH_INFO: `/v3/services` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:58.999486 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae8bcc8-f205-4fcc-842e-beaf0c94a85e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:58.999794 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae8bcc8-f205-4fcc-842e-beaf0c94a85e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:59.001153 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae8bcc8-f205-4fcc-842e-beaf0c94a85e None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:59.007140 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 466/922] 10.222.0.102 () {56 vars in 1127 bytes} [Thu Sep 1 17:18:58 2022] GET /identity/v3/services?name=object-store => generated 130 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:59.014701 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba8717b-434a-4939-804e-5be1b17bf2b0 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:59.116097 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5f3d08-9c2d-4a06-9fa0-48fe8c401d8d None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:59.116530 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5f3d08-9c2d-4a06-9fa0-48fe8c401d8d None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:59.116770 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5f3d08-9c2d-4a06-9fa0-48fe8c401d8d None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:59.118286 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 467/923] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:18:59 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:18:59.123473 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53aa1a5-e5ef-449a-af56-7d1ddbfec5cd None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:59.123763 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53aa1a5-e5ef-449a-af56-7d1ddbfec5cd None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:59.124039 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53aa1a5-e5ef-449a-af56-7d1ddbfec5cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:59.125487 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba8717b-434a-4939-804e-5be1b17bf2b0 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:59.143840 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-a53aa1a5-e5ef-449a-af56-7d1ddbfec5cd None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:18:59.145070 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-a53aa1a5-e5ef-449a-af56-7d1ddbfec5cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:18:59.162113 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e869c8-bbd5-456c-946a-450ef2bee0f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:59.163207 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e869c8-bbd5-456c-946a-450ef2bee0f4 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:59.163686 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e869c8-bbd5-456c-946a-450ef2bee0f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:59.164097 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e869c8-bbd5-456c-946a-450ef2bee0f4 None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:59.164861 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e869c8-bbd5-456c-946a-450ef2bee0f4 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:59.165322 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e869c8-bbd5-456c-946a-450ef2bee0f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:59.166853 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e869c8-bbd5-456c-946a-450ef2bee0f4 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:59.172322 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 457/924] 10.222.0.102 () {56 vars in 1127 bytes} [Thu Sep 1 17:18:59 2022] GET /identity/v3/services?type=object-store => generated 358 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:59.179137 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e869c8-bbd5-456c-946a-450ef2bee0f4 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:59.207499 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:59.220609 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 468/925] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:18:59 2022] POST /identity/v3/auth/tokens => generated 1476 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:18:59.222992 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f3fb94-bb3b-4156-8abf-9677e2d6a670 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:59.223539 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f3fb94-bb3b-4156-8abf-9677e2d6a670 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:59.224007 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f3fb94-bb3b-4156-8abf-9677e2d6a670 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:59.230523 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 469/926] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:18:59 2022] GET /identity => generated 271 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:18:59.231453 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93bbdb8-fdce-4c72-b03d-2d3576ffd529 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:59.231736 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93bbdb8-fdce-4c72-b03d-2d3576ffd529 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:59.232012 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93bbdb8-fdce-4c72-b03d-2d3576ffd529 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:59.254219 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-f93bbdb8-fdce-4c72-b03d-2d3576ffd529 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:18:59.255421 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-f93bbdb8-fdce-4c72-b03d-2d3576ffd529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:18:59.310620 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e869c8-bbd5-456c-946a-450ef2bee0f4 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:59.314969 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:59.337733 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 470/927] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:18:59 2022] POST /identity/v3/auth/tokens => generated 1476 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:18:59.359668 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7433c53-df88-4e38-b043-4d0e40b71ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:59.360840 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7433c53-df88-4e38-b043-4d0e40b71ebd None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:59.361492 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7433c53-df88-4e38-b043-4d0e40b71ebd None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:59.361946 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7433c53-df88-4e38-b043-4d0e40b71ebd None admin] PATH_INFO: `/v3/endpoints` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:59.362823 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7433c53-df88-4e38-b043-4d0e40b71ebd None admin] RBAC: Adding query filter params (interface=public, service_id=2cfefc081df04b67a52fd276f3c8eb5e, region_id=RegionOne) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:59.363284 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7433c53-df88-4e38-b043-4d0e40b71ebd None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:59.364797 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7433c53-df88-4e38-b043-4d0e40b71ebd None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:59.370612 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 458/928] 10.222.0.102 () {56 vars in 1256 bytes} [Thu Sep 1 17:18:59 2022] GET /identity/v3/endpoints?service_id=2cfefc081df04b67a52fd276f3c8eb5e&interface=public®ion_id=RegionOne => generated 195 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:18:59.376257 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93bbdb8-fdce-4c72-b03d-2d3576ffd529 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:59.486318 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93bbdb8-fdce-4c72-b03d-2d3576ffd529 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:59.497022 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7433c53-df88-4e38-b043-4d0e40b71ebd None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:59.528454 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ee8157-bc75-491e-ad73-8fab67da5cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:59.529206 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ee8157-bc75-491e-ad73-8fab67da5cf1 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:59.529519 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ee8157-bc75-491e-ad73-8fab67da5cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:59.529788 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ee8157-bc75-491e-ad73-8fab67da5cf1 None admin] PATH_INFO: `/v3/services` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:59.530386 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ee8157-bc75-491e-ad73-8fab67da5cf1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:59.531738 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ee8157-bc75-491e-ad73-8fab67da5cf1 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:59.539392 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 471/929] 10.222.0.102 () {56 vars in 1092 bytes} [Thu Sep 1 17:18:59 2022] GET /identity/v3/services => generated 1504 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:18:59.600908 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:59.615788 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7433c53-df88-4e38-b043-4d0e40b71ebd None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:59.655694 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba1b162-e533-4613-877c-85a745b9aec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:59.656341 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba1b162-e533-4613-877c-85a745b9aec9 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:59.656623 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba1b162-e533-4613-877c-85a745b9aec9 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:59.656897 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba1b162-e533-4613-877c-85a745b9aec9 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:59.666946 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ee8157-bc75-491e-ad73-8fab67da5cf1 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:59.668781 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba1b162-e533-4613-877c-85a745b9aec9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:18:59.675837 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-dba1b162-e533-4613-877c-85a745b9aec9 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 17:18:59.675837 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:59.675837 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:59.675837 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:59.675837 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:59.675837 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:59.675837 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:59.675837 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:59.675837 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:59.675837 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:59.675837 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:59.675837 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:59.675837 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:18:59.675837 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:18:59.675837 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:18:59.675837 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:18:59.676861 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:18:59.676861 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:18:59.676861 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:18:59.676861 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:18:59.676861 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:59.676861 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:59.676861 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:59.676861 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:59.676861 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:59.676861 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:59.676861 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:59.676861 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:59.676861 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:59.676861 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:59.676861 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:59.676861 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:59.677779 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:59.677779 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:59.677779 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:59.677779 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:59.677779 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:18:59.677779 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:18:59.677779 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:18:59.677779 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:18:59.677779 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:18:59.677779 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:18:59.677779 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 17:18:59.677779 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:18:59.678426 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 459/930] 10.222.0.102 () {56 vars in 1125 bytes} [Thu Sep 1 17:18:59 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:59.684262 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba1b162-e533-4613-877c-85a745b9aec9 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:59.740472 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:18:59.818555 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ee8157-bc75-491e-ad73-8fab67da5cf1 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:59.865348 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba1b162-e533-4613-877c-85a745b9aec9 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:18:59.876138 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3474fe1-7ee1-4a89-b6d0-a58ec2d4008c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:59.876474 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3474fe1-7ee1-4a89-b6d0-a58ec2d4008c None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:59.876754 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3474fe1-7ee1-4a89-b6d0-a58ec2d4008c None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:59.877027 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3474fe1-7ee1-4a89-b6d0-a58ec2d4008c None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:59.878138 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3474fe1-7ee1-4a89-b6d0-a58ec2d4008c None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:59.879500 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3474fe1-7ee1-4a89-b6d0-a58ec2d4008c None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:59.888671 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-e3474fe1-7ee1-4a89-b6d0-a58ec2d4008c None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 17:18:59.888671 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:59.888671 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 17:18:59.888671 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 17:18:59.888671 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 17:18:59.888671 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 17:18:59.888671 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 17:18:59.888671 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:18:59.888671 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 17:18:59.888671 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:18:59.888671 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 17:18:59.888671 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:18:59.888671 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:18:59.888671 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:18:59.888671 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:18:59.888671 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:18:59.889777 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:18:59.889777 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:18:59.889777 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:18:59.889777 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:18:59.889777 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:18:59.889777 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:18:59.889777 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:18:59.889777 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 17:18:59.889777 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 17:18:59.889777 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 17:18:59.889777 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 17:18:59.889777 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:18:59.889777 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:18:59.889777 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:18:59.889777 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:18:59.889777 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:18:59.889777 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:18:59.890646 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:18:59.890646 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:18:59.890646 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:18:59.890646 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:18:59.890646 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:18:59.890646 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:18:59.890646 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:18:59.890646 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:18:59.890646 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:18:59.890646 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:18:59.890646 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 17:18:59.890646 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:18:59.890646 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 17:18:59.890646 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:18:59.893672 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 472/931] 10.222.0.102 () {56 vars in 1137 bytes} [Thu Sep 1 17:18:59 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:18:59.896614 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3474fe1-7ee1-4a89-b6d0-a58ec2d4008c None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:59.909517 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7863a57f-67d2-47bb-ad1e-b113f1ab51c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:18:59.910491 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7863a57f-67d2-47bb-ad1e-b113f1ab51c8 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:18:59.910973 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7863a57f-67d2-47bb-ad1e-b113f1ab51c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:18:59.911459 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7863a57f-67d2-47bb-ad1e-b113f1ab51c8 None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:18:59.912694 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7863a57f-67d2-47bb-ad1e-b113f1ab51c8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:18:59.913360 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7863a57f-67d2-47bb-ad1e-b113f1ab51c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:18:59.915022 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7863a57f-67d2-47bb-ad1e-b113f1ab51c8 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:18:59.926557 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 460/932] 10.222.0.102 () {56 vars in 1120 bytes} [Thu Sep 1 17:18:59 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 340 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:18:59.934463 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7863a57f-67d2-47bb-ad1e-b113f1ab51c8 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:18:59.957049 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:00.011343 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3474fe1-7ee1-4a89-b6d0-a58ec2d4008c None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:00.029703 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:00.060047 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331fd2e3-1bfd-46e3-bad5-4119f49cf70a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:00.061343 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331fd2e3-1bfd-46e3-bad5-4119f49cf70a None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:00.062102 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331fd2e3-1bfd-46e3-bad5-4119f49cf70a None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:00.062599 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331fd2e3-1bfd-46e3-bad5-4119f49cf70a None admin] PATH_INFO: `/v3/services` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:00.063863 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331fd2e3-1bfd-46e3-bad5-4119f49cf70a None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:19:00.064398 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331fd2e3-1bfd-46e3-bad5-4119f49cf70a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:00.068358 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331fd2e3-1bfd-46e3-bad5-4119f49cf70a None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:00.085366 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7863a57f-67d2-47bb-ad1e-b113f1ab51c8 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:00.086194 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 473/933] 10.222.0.102 () {56 vars in 1131 bytes} [Thu Sep 1 17:18:59 2022] GET /identity/v3/services?name=compute_legacy => generated 132 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:19:00.088712 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331fd2e3-1bfd-46e3-bad5-4119f49cf70a None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:00.125003 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df33457-de93-41df-9e46-1901716dcfbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:00.126086 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df33457-de93-41df-9e46-1901716dcfbd None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:00.126648 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df33457-de93-41df-9e46-1901716dcfbd None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:00.127064 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df33457-de93-41df-9e46-1901716dcfbd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:00.142086 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df33457-de93-41df-9e46-1901716dcfbd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:00.143025 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df33457-de93-41df-9e46-1901716dcfbd None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:00.158284 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-5df33457-de93-41df-9e46-1901716dcfbd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:19:00.158284 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:19:00.158284 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:19:00.158284 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:19:00.158284 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:19:00.158284 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:19:00.158284 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:19:00.158284 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:19:00.158284 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:19:00.158284 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:19:00.158284 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:19:00.158284 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:19:00.158284 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:19:00.158284 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:19:00.158284 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:19:00.158284 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:19:00.159611 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:19:00.159611 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:19:00.159611 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:19:00.159611 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:19:00.159611 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:19:00.159611 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:19:00.159611 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:19:00.159611 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:19:00.159611 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:19:00.159611 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:19:00.159611 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:19:00.159611 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:19:00.159611 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:19:00.159611 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:19:00.159611 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:19:00.159611 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:19:00.159611 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:19:00.160832 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:19:00.160832 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:19:00.160832 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:19:00.165558 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 461/934] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:18:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:19:00.167837 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df33457-de93-41df-9e46-1901716dcfbd None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:00.232369 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331fd2e3-1bfd-46e3-bad5-4119f49cf70a None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:00.237598 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:00.271856 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd42f7c-3223-47d4-bd1e-c659154dd2dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:00.272578 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd42f7c-3223-47d4-bd1e-c659154dd2dd None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:00.272788 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd42f7c-3223-47d4-bd1e-c659154dd2dd None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:00.273026 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd42f7c-3223-47d4-bd1e-c659154dd2dd None admin] PATH_INFO: `/v3/services` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:00.273699 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd42f7c-3223-47d4-bd1e-c659154dd2dd None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:19:00.273998 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd42f7c-3223-47d4-bd1e-c659154dd2dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:00.275354 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd42f7c-3223-47d4-bd1e-c659154dd2dd None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:00.285525 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 474/935] 10.222.0.102 () {56 vars in 1131 bytes} [Thu Sep 1 17:19:00 2022] GET /identity/v3/services?type=compute_legacy => generated 388 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:19:00.287728 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd42f7c-3223-47d4-bd1e-c659154dd2dd None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:00.300966 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df33457-de93-41df-9e46-1901716dcfbd None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:00.339605 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:00.349280 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bedb499-2530-4861-8c82-053f0a5944c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:00.350139 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bedb499-2530-4861-8c82-053f0a5944c9 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:00.350863 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bedb499-2530-4861-8c82-053f0a5944c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:00.351300 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bedb499-2530-4861-8c82-053f0a5944c9 None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:00.352350 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bedb499-2530-4861-8c82-053f0a5944c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:19:00.353519 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bedb499-2530-4861-8c82-053f0a5944c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:00.355589 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bedb499-2530-4861-8c82-053f0a5944c9 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:00.372630 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 462/936] 10.222.0.102 () {56 vars in 1114 bytes} [Thu Sep 1 17:19:00 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:19:00.380161 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bedb499-2530-4861-8c82-053f0a5944c9 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:00.414603 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd42f7c-3223-47d4-bd1e-c659154dd2dd None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:00.454229 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18408bc7-11f7-45d3-814c-23ce61c942e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:00.454838 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18408bc7-11f7-45d3-814c-23ce61c942e7 None admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:00.455119 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18408bc7-11f7-45d3-814c-23ce61c942e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:00.455419 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18408bc7-11f7-45d3-814c-23ce61c942e7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:00.456127 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18408bc7-11f7-45d3-814c-23ce61c942e7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:00.457848 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18408bc7-11f7-45d3-814c-23ce61c942e7 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:00.485520 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 475/937] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:19:00 2022] POST /identity/v3/endpoints => generated 349 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:19:00.496922 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bedb499-2530-4861-8c82-053f0a5944c9 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:00.524934 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:00.545059 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d18884c-e24a-4954-b003-e52f66e8f3f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:00.545816 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d18884c-e24a-4954-b003-e52f66e8f3f5 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:00.546093 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d18884c-e24a-4954-b003-e52f66e8f3f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:00.546363 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d18884c-e24a-4954-b003-e52f66e8f3f5 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:00.555429 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d18884c-e24a-4954-b003-e52f66e8f3f5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:00.556908 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d18884c-e24a-4954-b003-e52f66e8f3f5 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:00.570496 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-8d18884c-e24a-4954-b003-e52f66e8f3f5 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 17:19:00.570496 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:19:00.570496 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:19:00.570496 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:19:00.570496 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:19:00.570496 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:19:00.570496 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:19:00.570496 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:19:00.570496 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:19:00.570496 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:19:00.570496 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:19:00.570496 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:19:00.570496 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:19:00.570496 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:19:00.570496 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:19:00.570496 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:19:00.571665 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:19:00.571665 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:19:00.571665 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:19:00.571665 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:19:00.571665 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:19:00.571665 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:19:00.571665 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:19:00.571665 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:19:00.571665 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:19:00.571665 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:19:00.571665 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:19:00.571665 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:19:00.571665 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:19:00.571665 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:19:00.571665 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:19:00.571665 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:19:00.571665 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:19:00.571665 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:19:00.573528 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:19:00.573528 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:19:00.573528 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:19:00.573528 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:19:00.573528 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:19:00.573528 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:19:00.573528 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:19:00.573528 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:19:00.573528 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 17:19:00.573528 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:19:00.573528 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 463/938] 10.222.0.102 () {56 vars in 1122 bytes} [Thu Sep 1 17:19:00 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:19:00.578030 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18408bc7-11f7-45d3-814c-23ce61c942e7 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:00.579774 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312f58c7-0e3f-495f-b278-68e5dd4b3e10 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:00.580050 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312f58c7-0e3f-495f-b278-68e5dd4b3e10 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:00.580344 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312f58c7-0e3f-495f-b278-68e5dd4b3e10 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:00.582053 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 464/939] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:19:00 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:19:00.587755 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f7930e-d844-4aca-b36e-f316f5337587 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:00.588057 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f7930e-d844-4aca-b36e-f316f5337587 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:00.588302 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f7930e-d844-4aca-b36e-f316f5337587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:00.609808 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-a2f7930e-d844-4aca-b36e-f316f5337587 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:19:00.611350 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-a2f7930e-d844-4aca-b36e-f316f5337587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:19:00.696039 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 465/940] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:19:00 2022] POST /identity/v3/auth/tokens => generated 1476 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:19:00.710488 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:00.714717 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18408bc7-11f7-45d3-814c-23ce61c942e7 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:00.757793 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f54e12-c4f0-4c30-a2f1-1f69cf13cda8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:00.758448 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f54e12-c4f0-4c30-a2f1-1f69cf13cda8 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:00.758731 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f54e12-c4f0-4c30-a2f1-1f69cf13cda8 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:00.759002 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f54e12-c4f0-4c30-a2f1-1f69cf13cda8 None admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:00.759722 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f54e12-c4f0-4c30-a2f1-1f69cf13cda8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:19:00.760023 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f54e12-c4f0-4c30-a2f1-1f69cf13cda8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:00.761458 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f54e12-c4f0-4c30-a2f1-1f69cf13cda8 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:00.780152 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 476/941] 10.222.0.102 () {56 vars in 1154 bytes} [Thu Sep 1 17:19:00 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 386 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:19:00.786436 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f7930e-d844-4aca-b36e-f316f5337587 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:00.907221 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f7930e-d844-4aca-b36e-f316f5337587 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:00.918548 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:00.951159 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa188f91-e887-491c-b180-17790a1de7b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:00.951897 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa188f91-e887-491c-b180-17790a1de7b7 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:00.952183 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa188f91-e887-491c-b180-17790a1de7b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:00.952455 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa188f91-e887-491c-b180-17790a1de7b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:00.963035 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa188f91-e887-491c-b180-17790a1de7b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:00.964496 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa188f91-e887-491c-b180-17790a1de7b7 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:00.980040 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-fa188f91-e887-491c-b180-17790a1de7b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:19:00.980040 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:19:00.980040 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:19:00.980040 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:19:00.980040 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:19:00.980040 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:19:00.980040 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:19:00.980040 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:19:00.980040 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:19:00.980040 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:19:00.980040 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:19:00.980040 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:19:00.980040 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:19:00.980040 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:19:00.980040 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:19:00.980040 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:19:00.981002 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:19:00.981002 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:19:00.981002 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:19:00.981002 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:19:00.981002 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:19:00.981002 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:19:00.981002 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:19:00.981002 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:19:00.981002 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:19:00.981002 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:19:00.981002 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:19:00.981002 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:19:00.981002 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:19:00.981002 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:19:00.981002 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:19:00.981002 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:19:00.981002 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:19:00.981972 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:19:00.981972 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:19:00.981972 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:19:00.981972 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 466/942] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:19:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:19:00.988058 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f54e12-c4f0-4c30-a2f1-1f69cf13cda8 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:00.994902 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa188f91-e887-491c-b180-17790a1de7b7 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:01.125867 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f54e12-c4f0-4c30-a2f1-1f69cf13cda8 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:01.129925 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa188f91-e887-491c-b180-17790a1de7b7 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:01.165030 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38418fcb-f61e-4b45-a80a-837cd20fc28f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:01.166043 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38418fcb-f61e-4b45-a80a-837cd20fc28f None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:01.166573 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38418fcb-f61e-4b45-a80a-837cd20fc28f None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:01.167058 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38418fcb-f61e-4b45-a80a-837cd20fc28f None admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:01.167910 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38418fcb-f61e-4b45-a80a-837cd20fc28f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:19:01.168417 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38418fcb-f61e-4b45-a80a-837cd20fc28f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:01.170113 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38418fcb-f61e-4b45-a80a-837cd20fc28f None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:01.173444 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b36cb97-4242-433b-a46c-e03f13567660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:01.174052 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b36cb97-4242-433b-a46c-e03f13567660 None admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:01.174970 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b36cb97-4242-433b-a46c-e03f13567660 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:01.175241 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b36cb97-4242-433b-a46c-e03f13567660 None admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:01.175882 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b36cb97-4242-433b-a46c-e03f13567660 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:01.177317 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b36cb97-4242-433b-a46c-e03f13567660 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:01.180980 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: WARNING keystone.server.flask.common [None req-1b36cb97-4242-433b-a46c-e03f13567660 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 01 17:19:01.185529 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 477/943] 10.222.0.102 () {56 vars in 1114 bytes} [Thu Sep 1 17:19:00 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:19:01.193374 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38418fcb-f61e-4b45-a80a-837cd20fc28f None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:01.207439 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.sql.core [None req-1b36cb97-4242-433b-a46c-e03f13567660 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 01 17:19:01.207439 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 01 17:19:01.207439 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [parameters: {'id': '64dca61519a945c7a2127b4b0d6eb1f2', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 01 17:19:01.207439 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81502) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 01 17:19:01.212904 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:01.219749 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-1b36cb97-4242-433b-a46c-e03f13567660 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 64dca61519a945c7a2127b4b0d6eb1f2.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 64dca61519a945c7a2127b4b0d6eb1f2. Sep 01 17:19:01.219749 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:19:01.219749 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 01 17:19:01.219749 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 01 17:19:01.219749 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 01 17:19:01.219749 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 01 17:19:01.219749 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 01 17:19:01.219749 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application result = self._query(query) Sep 01 17:19:01.219749 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 01 17:19:01.219749 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application conn.query(q) Sep 01 17:19:01.219749 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 01 17:19:01.219749 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 01 17:19:01.219749 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 01 17:19:01.220722 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application result.read() Sep 01 17:19:01.220722 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 01 17:19:01.220722 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 01 17:19:01.220722 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 01 17:19:01.220722 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 01 17:19:01.220722 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 01 17:19:01.220722 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 01 17:19:01.220722 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 01 17:19:01.220722 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 01 17:19:01.220722 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 01 17:19:01.220722 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:19:01.220722 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 01 17:19:01.220722 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:19:01.220722 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:19:01.220722 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 01 17:19:01.220722 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 01 17:19:01.220722 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 01 17:19:01.220722 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 01 17:19:01.222505 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 01 17:19:01.222505 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application next(self.gen) Sep 01 17:19:01.222505 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 01 17:19:01.222505 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application yield resource Sep 01 17:19:01.222505 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 01 17:19:01.222505 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application next(self.gen) Sep 01 17:19:01.222505 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 01 17:19:01.222505 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self.session.rollback() Sep 01 17:19:01.222505 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 01 17:19:01.222505 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self.force_reraise() Sep 01 17:19:01.222505 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 01 17:19:01.222505 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise self.value Sep 01 17:19:01.222505 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 01 17:19:01.222505 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 01 17:19:01.222505 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 01 17:19:01.222505 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application session.commit() Sep 01 17:19:01.222505 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 01 17:19:01.223583 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 01 17:19:01.223583 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 01 17:19:01.223583 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._prepare_impl() Sep 01 17:19:01.223583 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 01 17:19:01.223583 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self.session.flush() Sep 01 17:19:01.223583 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 01 17:19:01.223583 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._flush(objects) Sep 01 17:19:01.223583 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 01 17:19:01.223583 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 01 17:19:01.223583 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 01 17:19:01.223583 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application compat.raise_( Sep 01 17:19:01.223583 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 01 17:19:01.223583 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception Sep 01 17:19:01.223583 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 01 17:19:01.223583 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application flush_context.execute() Sep 01 17:19:01.223583 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 01 17:19:01.223583 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rec.execute(self) Sep 01 17:19:01.224733 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 01 17:19:01.224733 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 01 17:19:01.224733 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 01 17:19:01.224733 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 01 17:19:01.224733 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 01 17:19:01.224733 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 01 17:19:01.224733 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 01 17:19:01.224733 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 01 17:19:01.224733 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 01 17:19:01.224733 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 01 17:19:01.224733 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 01 17:19:01.224733 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 01 17:19:01.224733 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 01 17:19:01.224733 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 01 17:19:01.224733 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 01 17:19:01.225626 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 01 17:19:01.225626 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 01 17:19:01.225626 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception Sep 01 17:19:01.225626 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 01 17:19:01.225626 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 01 17:19:01.225626 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 01 17:19:01.225626 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 01 17:19:01.225626 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 01 17:19:01.225626 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application result = self._query(query) Sep 01 17:19:01.225626 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 01 17:19:01.225626 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application conn.query(q) Sep 01 17:19:01.225626 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 01 17:19:01.225626 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 01 17:19:01.225626 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 01 17:19:01.225626 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application result.read() Sep 01 17:19:01.225626 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 01 17:19:01.225626 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 01 17:19:01.226453 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 01 17:19:01.226453 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 01 17:19:01.226453 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 01 17:19:01.226453 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 01 17:19:01.226453 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 01 17:19:01.226453 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 01 17:19:01.226453 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 01 17:19:01.226453 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 01 17:19:01.226453 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [parameters: {'id': '64dca61519a945c7a2127b4b0d6eb1f2', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 01 17:19:01.226453 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 01 17:19:01.226453 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:19:01.226453 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 17:19:01.226453 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:19:01.226453 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:19:01.226453 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 01 17:19:01.227228 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 01 17:19:01.227228 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 01 17:19:01.227228 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 01 17:19:01.227228 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 01 17:19:01.227228 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:19:01.227228 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 17:19:01.227228 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:19:01.227228 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:19:01.227228 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:19:01.227228 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:19:01.227228 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:19:01.227228 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:19:01.227228 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:19:01.227228 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:19:01.227228 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:19:01.227228 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:19:01.227228 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:19:01.228467 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:19:01.228467 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 01 17:19:01.228467 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 01 17:19:01.228467 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:19:01.228467 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:19:01.228467 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 01 17:19:01.228467 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 01 17:19:01.228467 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 64dca61519a945c7a2127b4b0d6eb1f2. Sep 01 17:19:01.228467 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:19:01.230910 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 467/944] 10.222.0.102 () {60 vars in 1145 bytes} [Thu Sep 1 17:19:00 2022] POST /identity/v3/projects => generated 262 bytes in 236 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:19:01.240546 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b36cb97-4242-433b-a46c-e03f13567660 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:01.265678 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:01.317694 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38418fcb-f61e-4b45-a80a-837cd20fc28f None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:01.358319 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b56c60-86e8-429a-b86d-29f169062861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:01.359217 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b56c60-86e8-429a-b86d-29f169062861 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:01.359628 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b56c60-86e8-429a-b86d-29f169062861 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:01.360092 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b56c60-86e8-429a-b86d-29f169062861 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:01.373049 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b56c60-86e8-429a-b86d-29f169062861 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:01.374677 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b56c60-86e8-429a-b86d-29f169062861 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:01.381800 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b36cb97-4242-433b-a46c-e03f13567660 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:01.387463 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-04b56c60-86e8-429a-b86d-29f169062861 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:19:01.387463 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:19:01.387463 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:19:01.387463 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:19:01.387463 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:19:01.387463 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:19:01.387463 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:19:01.387463 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:19:01.387463 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:19:01.387463 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:19:01.387463 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:19:01.387463 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:19:01.387463 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:19:01.387463 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:19:01.387463 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:19:01.389707 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:19:01.389707 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:19:01.389707 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:19:01.389707 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:19:01.389707 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:19:01.389707 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:19:01.389707 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:19:01.389707 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:19:01.389707 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:19:01.389707 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:19:01.389707 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:19:01.389707 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:19:01.389707 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:19:01.389707 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:19:01.389707 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:19:01.389707 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:19:01.389707 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:19:01.391060 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:19:01.391060 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:19:01.391060 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:19:01.391060 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:19:01.391060 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:19:01.391060 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:19:01.391060 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:19:01.391060 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:19:01.391060 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 478/945] 10.222.0.102 () {56 vars in 1116 bytes} [Thu Sep 1 17:19:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:19:01.397075 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b56c60-86e8-429a-b86d-29f169062861 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:01.429846 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55f147c-c309-430b-9254-c4c75edb3b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:01.430972 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55f147c-c309-430b-9254-c4c75edb3b09 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:01.431319 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55f147c-c309-430b-9254-c4c75edb3b09 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:01.431603 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55f147c-c309-430b-9254-c4c75edb3b09 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:01.441475 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55f147c-c309-430b-9254-c4c75edb3b09 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:01.444373 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55f147c-c309-430b-9254-c4c75edb3b09 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:01.460169 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-e55f147c-c309-430b-9254-c4c75edb3b09 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:19:01.460169 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:19:01.460169 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:19:01.460169 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:19:01.460169 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:19:01.460169 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:19:01.460169 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:19:01.460169 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:19:01.460169 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:19:01.460169 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:19:01.460169 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:19:01.460169 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:19:01.460169 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:19:01.460169 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:19:01.460169 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:19:01.461115 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:19:01.461115 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:19:01.461115 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:19:01.461115 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:19:01.461115 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:19:01.461115 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:19:01.461115 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:19:01.461115 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:19:01.461115 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:19:01.461115 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:19:01.461115 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:19:01.461115 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:19:01.461115 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:19:01.461115 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:19:01.461115 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:19:01.461115 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:19:01.461115 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:19:01.462034 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:19:01.462034 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:19:01.462034 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:19:01.462034 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:19:01.462034 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:19:01.462034 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:19:01.462034 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:19:01.462034 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:19:01.465385 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 468/946] 10.222.0.102 () {56 vars in 1116 bytes} [Thu Sep 1 17:19:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:19:01.467414 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55f147c-c309-430b-9254-c4c75edb3b09 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:01.507623 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:01.538712 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b56c60-86e8-429a-b86d-29f169062861 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:01.567135 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:01.585237 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3270db-b573-4c6a-8cac-0adc59a8951d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:01.586072 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3270db-b573-4c6a-8cac-0adc59a8951d None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:01.586449 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3270db-b573-4c6a-8cac-0adc59a8951d None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:01.586844 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3270db-b573-4c6a-8cac-0adc59a8951d None admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:01.587736 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3270db-b573-4c6a-8cac-0adc59a8951d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:19:01.588198 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3270db-b573-4c6a-8cac-0adc59a8951d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:01.589847 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3270db-b573-4c6a-8cac-0adc59a8951d None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:01.594691 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55f147c-c309-430b-9254-c4c75edb3b09 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:01.603401 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 479/947] 10.222.0.102 () {56 vars in 1153 bytes} [Thu Sep 1 17:19:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:19:01.603401 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce33d1f-a26f-41f0-9235-bba3e549f3f9 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:01.603401 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce33d1f-a26f-41f0-9235-bba3e549f3f9 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:01.603401 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce33d1f-a26f-41f0-9235-bba3e549f3f9 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:01.606962 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 480/948] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:19:01 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:19:01.612169 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce33d1f-a26f-41f0-9235-bba3e549f3f9 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:01.636581 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0139fe9d-ea6b-4dc0-8b58-36cba910a0b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:01.637111 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0139fe9d-ea6b-4dc0-8b58-36cba910a0b3 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:01.637419 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0139fe9d-ea6b-4dc0-8b58-36cba910a0b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:01.637690 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0139fe9d-ea6b-4dc0-8b58-36cba910a0b3 None admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:01.638738 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0139fe9d-ea6b-4dc0-8b58-36cba910a0b3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:19:01.638738 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0139fe9d-ea6b-4dc0-8b58-36cba910a0b3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:01.640104 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0139fe9d-ea6b-4dc0-8b58-36cba910a0b3 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:01.667595 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 469/949] 10.222.0.102 () {56 vars in 1117 bytes} [Thu Sep 1 17:19:01 2022] GET /identity/v3/projects?name=service => generated 413 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:19:01.667595 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351b23b2-0f4f-49db-9f38-eb38f6ae6d3d None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:01.667595 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351b23b2-0f4f-49db-9f38-eb38f6ae6d3d None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:01.667595 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351b23b2-0f4f-49db-9f38-eb38f6ae6d3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:01.683710 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-351b23b2-0f4f-49db-9f38-eb38f6ae6d3d None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:19:01.685284 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-351b23b2-0f4f-49db-9f38-eb38f6ae6d3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:19:01.759215 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce33d1f-a26f-41f0-9235-bba3e549f3f9 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:01.776422 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 470/950] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:19:01 2022] POST /identity/v3/auth/tokens => generated 1476 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:19:01.806567 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdd5112-04a7-4ed6-8672-17f7eed8bd6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:01.807660 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdd5112-04a7-4ed6-8672-17f7eed8bd6b None admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:01.808180 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdd5112-04a7-4ed6-8672-17f7eed8bd6b None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:01.808659 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdd5112-04a7-4ed6-8672-17f7eed8bd6b None admin] PATH_INFO: `/v3/projects/e78fff7f1f6e4f68b8d233ad2e5bbd4f/users/575a76b99ea945efb1024fbb3dc81aaf/roles/9329b00eca174ca5a29094e84e243fa4` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:01.811304 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdd5112-04a7-4ed6-8672-17f7eed8bd6b None admin] RBAC: Authorizing `identity:create_grant(project_id=e78fff7f1f6e4f68b8d233ad2e5bbd4f, user_id=575a76b99ea945efb1024fbb3dc81aaf, role_id=9329b00eca174ca5a29094e84e243fa4)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:01.817095 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdd5112-04a7-4ed6-8672-17f7eed8bd6b None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:01.824504 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:01.832452 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 481/951] 10.222.0.102 () {58 vars in 1444 bytes} [Thu Sep 1 17:19:01 2022] PUT /identity/v3/projects/e78fff7f1f6e4f68b8d233ad2e5bbd4f/users/575a76b99ea945efb1024fbb3dc81aaf/roles/9329b00eca174ca5a29094e84e243fa4 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:19:02.067922 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351b23b2-0f4f-49db-9f38-eb38f6ae6d3d None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:02.138073 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:02.175678 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351b23b2-0f4f-49db-9f38-eb38f6ae6d3d None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:02.212747 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5735f1d6-595c-4f57-a8de-5be5a9b54d04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:02.213425 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5735f1d6-595c-4f57-a8de-5be5a9b54d04 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:02.213712 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5735f1d6-595c-4f57-a8de-5be5a9b54d04 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:02.213987 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5735f1d6-595c-4f57-a8de-5be5a9b54d04 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:02.214585 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5735f1d6-595c-4f57-a8de-5be5a9b54d04 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:02.215967 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5735f1d6-595c-4f57-a8de-5be5a9b54d04 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:02.225205 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-5735f1d6-595c-4f57-a8de-5be5a9b54d04 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 17:19:02.225205 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:19:02.225205 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 17:19:02.225205 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 17:19:02.225205 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 17:19:02.225205 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 17:19:02.225205 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 17:19:02.225205 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:19:02.225205 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 17:19:02.225205 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:19:02.225205 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 17:19:02.225205 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:19:02.225205 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:19:02.225205 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:19:02.225205 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:19:02.225205 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:19:02.225205 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:19:02.226251 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:19:02.226251 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:19:02.226251 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:19:02.226251 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:19:02.226251 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:19:02.226251 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:19:02.226251 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 17:19:02.226251 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 17:19:02.226251 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 17:19:02.226251 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 17:19:02.226251 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:19:02.226251 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:19:02.226251 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:19:02.226251 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:19:02.226251 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:19:02.226251 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:19:02.227107 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:19:02.227107 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:19:02.227107 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:19:02.227107 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:19:02.227107 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:19:02.227107 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:19:02.227107 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:19:02.227107 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:19:02.227107 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:19:02.227107 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:19:02.227107 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 17:19:02.227107 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:19:02.227107 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 17:19:02.227107 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:19:02.227107 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 471/952] 10.222.0.102 () {56 vars in 1131 bytes} [Thu Sep 1 17:19:02 2022] GET /identity/v3/services/object-store => generated 93 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:19:02.234977 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdd5112-04a7-4ed6-8672-17f7eed8bd6b None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:02.344113 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdd5112-04a7-4ed6-8672-17f7eed8bd6b None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:02.380557 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c4a022-cb87-43a1-8e16-afe1ee5aa966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:02.381181 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c4a022-cb87-43a1-8e16-afe1ee5aa966 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:02.381488 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c4a022-cb87-43a1-8e16-afe1ee5aa966 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:02.381763 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c4a022-cb87-43a1-8e16-afe1ee5aa966 None admin] PATH_INFO: `/v3/services` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:02.382401 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c4a022-cb87-43a1-8e16-afe1ee5aa966 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:19:02.382707 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c4a022-cb87-43a1-8e16-afe1ee5aa966 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:02.384068 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c4a022-cb87-43a1-8e16-afe1ee5aa966 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:02.389882 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 482/953] 10.222.0.102 () {56 vars in 1127 bytes} [Thu Sep 1 17:19:02 2022] GET /identity/v3/services?name=object-store => generated 130 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:19:02.395863 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5735f1d6-595c-4f57-a8de-5be5a9b54d04 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:02.503356 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5735f1d6-595c-4f57-a8de-5be5a9b54d04 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:02.537738 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:02.539469 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1baa869-5908-4515-9ad7-2a1b314347dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:02.540104 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1baa869-5908-4515-9ad7-2a1b314347dd None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:02.540579 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1baa869-5908-4515-9ad7-2a1b314347dd None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:02.540956 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1baa869-5908-4515-9ad7-2a1b314347dd None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:02.541883 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1baa869-5908-4515-9ad7-2a1b314347dd None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:19:02.542252 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1baa869-5908-4515-9ad7-2a1b314347dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:02.543862 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1baa869-5908-4515-9ad7-2a1b314347dd None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:02.550215 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 472/954] 10.222.0.102 () {56 vars in 1127 bytes} [Thu Sep 1 17:19:02 2022] GET /identity/v3/services?type=object-store => generated 358 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:19:02.555751 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c4a022-cb87-43a1-8e16-afe1ee5aa966 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:02.581041 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c778a1e-d67e-4620-8750-f6642021f33d None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:02.581452 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c778a1e-d67e-4620-8750-f6642021f33d None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:02.581734 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c778a1e-d67e-4620-8750-f6642021f33d None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:02.583329 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 473/955] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:19:02 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Sep 01 17:19:02.588100 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58d341f-1528-4eb1-a704-1d70945dc0dd None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:02.588379 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58d341f-1528-4eb1-a704-1d70945dc0dd None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:02.588651 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58d341f-1528-4eb1-a704-1d70945dc0dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:02.609226 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-c58d341f-1528-4eb1-a704-1d70945dc0dd None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:19:02.610353 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-c58d341f-1528-4eb1-a704-1d70945dc0dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:19:02.637639 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:02.688489 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c4a022-cb87-43a1-8e16-afe1ee5aa966 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:02.691126 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 474/956] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:19:02 2022] POST /identity/v3/auth/tokens => generated 1476 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 17:19:02.724651 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0cc908-7f78-48d5-bfc6-63161bc39dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:02.725273 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0cc908-7f78-48d5-bfc6-63161bc39dd0 None admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:02.725585 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0cc908-7f78-48d5-bfc6-63161bc39dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:02.725867 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0cc908-7f78-48d5-bfc6-63161bc39dd0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:02.726542 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0cc908-7f78-48d5-bfc6-63161bc39dd0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:02.728173 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0cc908-7f78-48d5-bfc6-63161bc39dd0 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:02.751942 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 483/957] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:19:02 2022] POST /identity/v3/endpoints => generated 351 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:19:02.970548 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58d341f-1528-4eb1-a704-1d70945dc0dd None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:03.077662 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58d341f-1528-4eb1-a704-1d70945dc0dd None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:03.104182 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:03.123063 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5b9894-c897-4122-bede-4ba0c8560fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:03.123700 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5b9894-c897-4122-bede-4ba0c8560fab None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:03.123975 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5b9894-c897-4122-bede-4ba0c8560fab None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:03.124308 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5b9894-c897-4122-bede-4ba0c8560fab None admin] PATH_INFO: `/v3/services/compute` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:03.124893 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5b9894-c897-4122-bede-4ba0c8560fab None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:03.126326 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5b9894-c897-4122-bede-4ba0c8560fab None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:03.135523 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-4c5b9894-c897-4122-bede-4ba0c8560fab None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 17:19:03.135523 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:19:03.135523 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 17:19:03.135523 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 17:19:03.135523 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 17:19:03.135523 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 17:19:03.135523 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 17:19:03.135523 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:19:03.135523 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 17:19:03.135523 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:19:03.135523 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 17:19:03.135523 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:19:03.135523 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:19:03.135523 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:19:03.135523 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:19:03.135523 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:19:03.135523 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:19:03.136579 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:19:03.136579 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:19:03.136579 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:19:03.136579 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:19:03.136579 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:19:03.136579 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:19:03.136579 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 17:19:03.136579 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 17:19:03.136579 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 17:19:03.136579 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 17:19:03.136579 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:19:03.136579 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:19:03.136579 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:19:03.136579 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:19:03.136579 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:19:03.136579 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:19:03.137476 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:19:03.137476 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:19:03.137476 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:19:03.137476 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:19:03.137476 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:19:03.137476 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:19:03.137476 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:19:03.137476 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:19:03.137476 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:19:03.137476 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:19:03.137476 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 17:19:03.137476 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:19:03.137476 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 17:19:03.137476 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:19:03.137476 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 475/958] 10.222.0.102 () {56 vars in 1116 bytes} [Thu Sep 1 17:19:02 2022] GET /identity/v3/services/compute => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:19:03.142699 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0cc908-7f78-48d5-bfc6-63161bc39dd0 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:03.249534 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0cc908-7f78-48d5-bfc6-63161bc39dd0 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:03.268557 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:03.287111 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78ac7f5-e23e-423d-9072-ebccefdc144d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:03.287726 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78ac7f5-e23e-423d-9072-ebccefdc144d None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:03.288010 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78ac7f5-e23e-423d-9072-ebccefdc144d None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:03.288284 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78ac7f5-e23e-423d-9072-ebccefdc144d None admin] PATH_INFO: `/v3/services` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:03.288917 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78ac7f5-e23e-423d-9072-ebccefdc144d None admin] RBAC: Adding query filter params (name=compute) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:19:03.289223 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78ac7f5-e23e-423d-9072-ebccefdc144d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:03.290622 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78ac7f5-e23e-423d-9072-ebccefdc144d None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:03.295982 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 484/959] 10.222.0.102 () {56 vars in 1117 bytes} [Thu Sep 1 17:19:03 2022] GET /identity/v3/services?name=compute => generated 125 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:19:03.301709 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5b9894-c897-4122-bede-4ba0c8560fab None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:03.413962 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5b9894-c897-4122-bede-4ba0c8560fab None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:03.463387 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559462a5-49a3-40af-b5b3-f03d118a1e65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:03.464277 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559462a5-49a3-40af-b5b3-f03d118a1e65 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:03.464698 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559462a5-49a3-40af-b5b3-f03d118a1e65 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:03.465110 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559462a5-49a3-40af-b5b3-f03d118a1e65 None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:03.466283 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559462a5-49a3-40af-b5b3-f03d118a1e65 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:19:03.466754 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559462a5-49a3-40af-b5b3-f03d118a1e65 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:03.468918 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559462a5-49a3-40af-b5b3-f03d118a1e65 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:03.474878 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 476/960] 10.222.0.102 () {56 vars in 1117 bytes} [Thu Sep 1 17:19:03 2022] GET /identity/v3/services?type=compute => generated 125 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:19:03.688438 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9216bb09-5fc8-4bf5-95d5-6be1aacd7698 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:03.689140 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9216bb09-5fc8-4bf5-95d5-6be1aacd7698 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:03.689140 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9216bb09-5fc8-4bf5-95d5-6be1aacd7698 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:03.691125 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 485/961] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:19:03 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:19:03.696401 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0baaa1f-14b8-41fe-94bf-9a0f8d1f441a None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:03.696958 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0baaa1f-14b8-41fe-94bf-9a0f8d1f441a None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:03.697466 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0baaa1f-14b8-41fe-94bf-9a0f8d1f441a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:03.716736 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-f0baaa1f-14b8-41fe-94bf-9a0f8d1f441a None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:19:03.718279 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-f0baaa1f-14b8-41fe-94bf-9a0f8d1f441a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:19:03.770914 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:03.804813 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 477/962] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:19:03 2022] POST /identity/v3/auth/tokens => generated 1476 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:19:03.953514 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5483f5a8-1018-4047-957d-698c17dbed43 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:03.954127 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5483f5a8-1018-4047-957d-698c17dbed43 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:03.954626 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5483f5a8-1018-4047-957d-698c17dbed43 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:03.956978 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 486/963] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:19:03 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:19:03.962532 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f661fa-0db9-4000-a7fe-78c0261c46dd None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:03.963054 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f661fa-0db9-4000-a7fe-78c0261c46dd None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:03.963552 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f661fa-0db9-4000-a7fe-78c0261c46dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:03.983126 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-e7f661fa-0db9-4000-a7fe-78c0261c46dd None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:19:03.984679 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-e7f661fa-0db9-4000-a7fe-78c0261c46dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:19:04.061425 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 478/964] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:19:03 2022] POST /identity/v3/auth/tokens => generated 1476 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:19:04.071428 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5483f5a8-1018-4047-957d-698c17dbed43 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:04.185528 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5483f5a8-1018-4047-957d-698c17dbed43 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:04.221762 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36db4296-f5f1-4a43-b8f5-62ac4393dfd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:04.222298 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36db4296-f5f1-4a43-b8f5-62ac4393dfd5 None admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:04.222578 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36db4296-f5f1-4a43-b8f5-62ac4393dfd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:04.222852 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36db4296-f5f1-4a43-b8f5-62ac4393dfd5 None admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:04.223501 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36db4296-f5f1-4a43-b8f5-62ac4393dfd5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:04.224929 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36db4296-f5f1-4a43-b8f5-62ac4393dfd5 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:04.227746 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: WARNING keystone.server.flask.common [None req-36db4296-f5f1-4a43-b8f5-62ac4393dfd5 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 01 17:19:04.242237 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.sql.core [None req-36db4296-f5f1-4a43-b8f5-62ac4393dfd5 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 01 17:19:04.242237 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 01 17:19:04.242237 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [parameters: {'user_id': '7a11fee77bf149969e105f4bea622bde', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 01 17:19:04.242237 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81501) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 01 17:19:04.259806 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-36db4296-f5f1-4a43-b8f5-62ac4393dfd5 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 01 17:19:04.259806 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:19:04.259806 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 01 17:19:04.259806 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 01 17:19:04.259806 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 01 17:19:04.259806 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 01 17:19:04.259806 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 01 17:19:04.259806 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application result = self._query(query) Sep 01 17:19:04.259806 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 01 17:19:04.259806 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application conn.query(q) Sep 01 17:19:04.259806 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 01 17:19:04.259806 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 01 17:19:04.259806 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 01 17:19:04.259806 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application result.read() Sep 01 17:19:04.259806 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 01 17:19:04.260700 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 01 17:19:04.260700 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 01 17:19:04.260700 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 01 17:19:04.260700 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 01 17:19:04.260700 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 01 17:19:04.260700 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 01 17:19:04.260700 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 01 17:19:04.260700 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 01 17:19:04.260700 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:19:04.260700 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 01 17:19:04.260700 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:19:04.260700 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:19:04.260700 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 01 17:19:04.260700 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 01 17:19:04.260700 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 01 17:19:04.260700 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 01 17:19:04.260700 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 01 17:19:04.260700 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application next(self.gen) Sep 01 17:19:04.261680 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 01 17:19:04.261680 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application yield resource Sep 01 17:19:04.261680 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 01 17:19:04.261680 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application next(self.gen) Sep 01 17:19:04.261680 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 01 17:19:04.261680 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application self.session.rollback() Sep 01 17:19:04.261680 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 01 17:19:04.261680 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application self.force_reraise() Sep 01 17:19:04.261680 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 01 17:19:04.261680 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise self.value Sep 01 17:19:04.261680 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 01 17:19:04.261680 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 01 17:19:04.261680 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 01 17:19:04.261680 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application session.commit() Sep 01 17:19:04.261680 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 01 17:19:04.261680 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 01 17:19:04.261680 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 01 17:19:04.262575 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application self._prepare_impl() Sep 01 17:19:04.262575 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 01 17:19:04.262575 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application self.session.flush() Sep 01 17:19:04.262575 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 01 17:19:04.262575 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application self._flush(objects) Sep 01 17:19:04.262575 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 01 17:19:04.262575 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 01 17:19:04.262575 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 01 17:19:04.262575 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application compat.raise_( Sep 01 17:19:04.262575 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 01 17:19:04.262575 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception Sep 01 17:19:04.262575 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 01 17:19:04.262575 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application flush_context.execute() Sep 01 17:19:04.262575 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 01 17:19:04.262575 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rec.execute(self) Sep 01 17:19:04.262575 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 01 17:19:04.262575 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 01 17:19:04.263444 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 01 17:19:04.263444 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 01 17:19:04.263444 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 01 17:19:04.263444 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 01 17:19:04.263444 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 01 17:19:04.263444 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 01 17:19:04.263444 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 01 17:19:04.263444 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 01 17:19:04.263444 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 01 17:19:04.263444 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 01 17:19:04.263444 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 01 17:19:04.263444 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 01 17:19:04.263444 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 01 17:19:04.263444 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 01 17:19:04.263444 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 01 17:19:04.263444 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception Sep 01 17:19:04.264295 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 01 17:19:04.264295 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 01 17:19:04.264295 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 01 17:19:04.264295 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 01 17:19:04.264295 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 01 17:19:04.264295 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application result = self._query(query) Sep 01 17:19:04.264295 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 01 17:19:04.264295 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application conn.query(q) Sep 01 17:19:04.264295 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 01 17:19:04.264295 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 01 17:19:04.264295 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 01 17:19:04.264295 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application result.read() Sep 01 17:19:04.264295 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 01 17:19:04.264295 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 01 17:19:04.264295 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 01 17:19:04.264295 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 01 17:19:04.265120 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 01 17:19:04.265120 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 01 17:19:04.265120 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 01 17:19:04.265120 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 01 17:19:04.265120 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 01 17:19:04.265120 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 01 17:19:04.265120 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [parameters: {'user_id': '7a11fee77bf149969e105f4bea622bde', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 01 17:19:04.265120 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 01 17:19:04.265120 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:19:04.265120 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 17:19:04.265120 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:19:04.265120 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:19:04.265120 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:19:04.265120 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:19:04.265120 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:19:04.266001 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:19:04.266001 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:19:04.266001 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:19:04.266001 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:19:04.266001 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:19:04.266001 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:19:04.266001 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:19:04.266001 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 01 17:19:04.266001 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 01 17:19:04.266001 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:19:04.266001 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:19:04.266001 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:19:04.266001 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:19:04.266001 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:19:04.266001 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:19:04.266001 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 01 17:19:04.266001 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 01 17:19:04.266917 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 01 17:19:04.266917 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 01 17:19:04.266917 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 01 17:19:04.266917 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 01 17:19:04.266917 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 01 17:19:04.266917 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:19:04.266917 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 487/965] 10.222.0.102 () {60 vars in 1136 bytes} [Thu Sep 1 17:19:04 2022] POST /identity/v3/users => generated 160 bytes in 190 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:19:04.272317 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f661fa-0db9-4000-a7fe-78c0261c46dd None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:04.368684 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36db4296-f5f1-4a43-b8f5-62ac4393dfd5 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:04.384291 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f661fa-0db9-4000-a7fe-78c0261c46dd None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:04.398095 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:04.428685 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c717f657-2f08-4bd1-be44-e56e6d9bde5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:04.429549 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c717f657-2f08-4bd1-be44-e56e6d9bde5c None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:04.429858 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c717f657-2f08-4bd1-be44-e56e6d9bde5c None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:04.430139 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c717f657-2f08-4bd1-be44-e56e6d9bde5c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:04.431334 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:04.439243 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c717f657-2f08-4bd1-be44-e56e6d9bde5c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:04.440699 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c717f657-2f08-4bd1-be44-e56e6d9bde5c None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:04.457461 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-c717f657-2f08-4bd1-be44-e56e6d9bde5c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 17:19:04.457461 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:19:04.457461 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:19:04.457461 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:19:04.457461 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:19:04.457461 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:19:04.457461 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:19:04.457461 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:19:04.457461 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:19:04.457461 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:19:04.457461 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:19:04.457461 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:19:04.457461 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:19:04.457461 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:19:04.457461 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:19:04.457461 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:19:04.458398 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:19:04.458398 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:19:04.458398 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:19:04.458398 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:19:04.458398 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:19:04.458398 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:19:04.458398 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:19:04.458398 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:19:04.458398 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:19:04.458398 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:19:04.458398 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:19:04.458398 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:19:04.458398 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:19:04.458398 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:19:04.458398 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:19:04.458398 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:19:04.458398 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:19:04.458398 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:19:04.459366 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:19:04.459366 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:19:04.459366 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:19:04.459366 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:19:04.459366 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:19:04.459366 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:19:04.459366 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:19:04.459366 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:19:04.459366 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 17:19:04.459366 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:19:04.464030 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 479/966] 10.222.0.102 () {56 vars in 1104 bytes} [Thu Sep 1 17:19:04 2022] GET /identity/v3/users/cinder => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:19:04.465642 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c717f657-2f08-4bd1-be44-e56e6d9bde5c None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:04.505437 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36db4296-f5f1-4a43-b8f5-62ac4393dfd5 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:04.538460 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce8e297-6507-43a4-9754-0055983253f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:04.539099 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce8e297-6507-43a4-9754-0055983253f6 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:04.539379 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce8e297-6507-43a4-9754-0055983253f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:04.539649 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce8e297-6507-43a4-9754-0055983253f6 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:04.551213 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce8e297-6507-43a4-9754-0055983253f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:19:04.556173 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-1ce8e297-6507-43a4-9754-0055983253f6 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 17:19:04.556173 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:19:04.556173 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:19:04.556173 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:19:04.556173 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:19:04.556173 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:19:04.556173 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:19:04.556173 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:19:04.556173 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:19:04.556173 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:19:04.556173 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:19:04.556173 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:19:04.556173 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:19:04.556173 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:19:04.556173 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:19:04.556173 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:19:04.557160 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:19:04.557160 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:19:04.557160 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:19:04.557160 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:19:04.557160 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:19:04.557160 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:19:04.557160 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:19:04.557160 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:19:04.557160 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:19:04.557160 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:19:04.557160 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:19:04.557160 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:19:04.557160 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:19:04.557160 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:19:04.557160 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:19:04.557160 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:19:04.558037 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:19:04.558037 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:19:04.558037 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:19:04.558037 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:19:04.558037 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:19:04.558037 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:19:04.558037 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:19:04.558037 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:19:04.558037 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:19:04.558037 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:19:04.558037 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 17:19:04.558037 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:19:04.558037 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 488/967] 10.222.0.102 () {56 vars in 1125 bytes} [Thu Sep 1 17:19:04 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:19:04.564452 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce8e297-6507-43a4-9754-0055983253f6 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:04.581527 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c717f657-2f08-4bd1-be44-e56e6d9bde5c None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:04.618183 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ace50c-90a0-47bf-915f-2712ff9f8663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:04.618809 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ace50c-90a0-47bf-915f-2712ff9f8663 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:04.619093 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ace50c-90a0-47bf-915f-2712ff9f8663 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:04.619369 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ace50c-90a0-47bf-915f-2712ff9f8663 None admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:04.620089 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ace50c-90a0-47bf-915f-2712ff9f8663 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:19:04.620398 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ace50c-90a0-47bf-915f-2712ff9f8663 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:04.621937 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ace50c-90a0-47bf-915f-2712ff9f8663 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:04.665464 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 480/968] 10.222.0.102 () {56 vars in 1106 bytes} [Thu Sep 1 17:19:04 2022] GET /identity/v3/users?name=cinder => generated 356 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:19:04.677887 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce8e297-6507-43a4-9754-0055983253f6 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:04.680912 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:04.720832 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4f7355-826a-4e67-9c52-edaf3cd5e53a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:04.721657 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4f7355-826a-4e67-9c52-edaf3cd5e53a None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:04.721817 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4f7355-826a-4e67-9c52-edaf3cd5e53a None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:04.721994 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4f7355-826a-4e67-9c52-edaf3cd5e53a None admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:04.722674 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4f7355-826a-4e67-9c52-edaf3cd5e53a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:19:04.722976 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4f7355-826a-4e67-9c52-edaf3cd5e53a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:04.724593 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4f7355-826a-4e67-9c52-edaf3cd5e53a None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:04.738252 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e43b293-d41a-48c4-a44d-b572516dad99 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:04.738584 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e43b293-d41a-48c4-a44d-b572516dad99 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:04.738817 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e43b293-d41a-48c4-a44d-b572516dad99 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:04.739700 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 489/969] 10.222.0.102 () {56 vars in 1120 bytes} [Thu Sep 1 17:19:04 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 340 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:19:04.740553 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 481/970] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:19:04 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:19:04.745330 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4f7355-826a-4e67-9c52-edaf3cd5e53a None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:04.745952 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84a9f71-6558-4adb-8491-ddd315a711f5 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:04.746249 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84a9f71-6558-4adb-8491-ddd315a711f5 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:04.746528 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84a9f71-6558-4adb-8491-ddd315a711f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:04.768860 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-f84a9f71-6558-4adb-8491-ddd315a711f5 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:19:04.770066 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-f84a9f71-6558-4adb-8491-ddd315a711f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:19:04.793934 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:04.864792 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 482/971] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:19:04 2022] POST /identity/v3/auth/tokens => generated 1476 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:19:04.881432 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4f7355-826a-4e67-9c52-edaf3cd5e53a None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:04.919586 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a3d620-f71d-4cdf-b01c-57e7cb8c9a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:04.920218 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a3d620-f71d-4cdf-b01c-57e7cb8c9a6c None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:04.920502 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a3d620-f71d-4cdf-b01c-57e7cb8c9a6c None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:04.920774 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a3d620-f71d-4cdf-b01c-57e7cb8c9a6c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:04.932260 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a3d620-f71d-4cdf-b01c-57e7cb8c9a6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:04.933937 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a3d620-f71d-4cdf-b01c-57e7cb8c9a6c None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:04.949658 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-c1a3d620-f71d-4cdf-b01c-57e7cb8c9a6c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:19:04.949658 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:19:04.949658 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:19:04.949658 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:19:04.949658 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:19:04.949658 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:19:04.949658 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:19:04.949658 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:19:04.949658 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:19:04.949658 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:19:04.949658 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:19:04.949658 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:19:04.949658 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:19:04.949658 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:19:04.949658 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:19:04.949658 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:19:04.950760 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:19:04.950760 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:19:04.950760 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:19:04.950760 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:19:04.950760 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:19:04.950760 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:19:04.950760 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:19:04.950760 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:19:04.950760 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:19:04.950760 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:19:04.950760 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:19:04.950760 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:19:04.950760 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:19:04.950760 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:19:04.950760 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:19:04.950760 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:19:04.950760 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:19:04.952019 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:19:04.952019 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:19:04.952019 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:19:04.952019 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 490/972] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:19:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:19:04.957169 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84a9f71-6558-4adb-8491-ddd315a711f5 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:05.068399 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84a9f71-6558-4adb-8491-ddd315a711f5 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:05.104829 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef7df60-3dde-432e-8229-a8c5d46afcd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:05.105470 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef7df60-3dde-432e-8229-a8c5d46afcd7 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:05.105749 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef7df60-3dde-432e-8229-a8c5d46afcd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:05.106016 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef7df60-3dde-432e-8229-a8c5d46afcd7 None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:05.106650 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef7df60-3dde-432e-8229-a8c5d46afcd7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:19:05.106948 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef7df60-3dde-432e-8229-a8c5d46afcd7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:05.108309 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef7df60-3dde-432e-8229-a8c5d46afcd7 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:05.124586 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 483/973] 10.222.0.102 () {56 vars in 1114 bytes} [Thu Sep 1 17:19:04 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:19:05.126072 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a3d620-f71d-4cdf-b01c-57e7cb8c9a6c None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:05.145045 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef7df60-3dde-432e-8229-a8c5d46afcd7 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:05.147890 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:05.165822 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:05.243566 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a3d620-f71d-4cdf-b01c-57e7cb8c9a6c None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:05.254787 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef7df60-3dde-432e-8229-a8c5d46afcd7 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:05.279270 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f91d6ba-c5b1-49aa-b3a6-2faa9803cabf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:05.279898 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f91d6ba-c5b1-49aa-b3a6-2faa9803cabf None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:05.280180 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f91d6ba-c5b1-49aa-b3a6-2faa9803cabf None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:05.280455 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f91d6ba-c5b1-49aa-b3a6-2faa9803cabf None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:05.288756 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f91d6ba-c5b1-49aa-b3a6-2faa9803cabf None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:05.290216 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f91d6ba-c5b1-49aa-b3a6-2faa9803cabf None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:05.292688 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1648b7-3f2f-41dc-a9d5-06ed8cc00baa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:05.293843 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1648b7-3f2f-41dc-a9d5-06ed8cc00baa None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:05.294117 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1648b7-3f2f-41dc-a9d5-06ed8cc00baa None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:05.294382 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1648b7-3f2f-41dc-a9d5-06ed8cc00baa None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:05.294962 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1648b7-3f2f-41dc-a9d5-06ed8cc00baa None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:05.296336 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1648b7-3f2f-41dc-a9d5-06ed8cc00baa None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:05.301920 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-0f91d6ba-c5b1-49aa-b3a6-2faa9803cabf None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 17:19:05.301920 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:19:05.301920 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:19:05.301920 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:19:05.301920 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:19:05.301920 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:19:05.301920 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:19:05.301920 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:19:05.301920 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:19:05.301920 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:19:05.301920 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:19:05.301920 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:19:05.301920 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:19:05.301920 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:19:05.301920 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:19:05.301920 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:19:05.302849 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:19:05.302849 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:19:05.302849 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:19:05.302849 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:19:05.302849 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:19:05.302849 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:19:05.302849 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:19:05.302849 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:19:05.302849 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:19:05.302849 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:19:05.302849 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:19:05.302849 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:19:05.302849 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:19:05.302849 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:19:05.302849 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:19:05.302849 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:19:05.302849 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:19:05.302849 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:19:05.308699 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:19:05.308699 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:19:05.308699 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:19:05.308699 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:19:05.308699 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:19:05.308699 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:19:05.308699 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:19:05.308699 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:19:05.308699 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 17:19:05.308699 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:19:05.308699 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 491/974] 10.222.0.102 () {56 vars in 1122 bytes} [Thu Sep 1 17:19:05 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:19:05.309338 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-0f1648b7-3f2f-41dc-a9d5-06ed8cc00baa None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 17:19:05.309338 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:19:05.309338 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 17:19:05.309338 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 17:19:05.309338 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 17:19:05.309338 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 17:19:05.309338 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 17:19:05.309338 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:19:05.309338 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 17:19:05.309338 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:19:05.309338 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 17:19:05.309338 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:19:05.309338 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:19:05.309338 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:19:05.309338 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:19:05.309338 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:19:05.309338 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:19:05.310231 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:19:05.310231 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:19:05.310231 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:19:05.310231 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:19:05.310231 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:19:05.310231 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:19:05.310231 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 17:19:05.310231 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 17:19:05.310231 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 17:19:05.310231 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 17:19:05.310231 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:19:05.310231 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:19:05.310231 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:19:05.310231 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:19:05.310231 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:19:05.310231 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:19:05.311120 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:19:05.311120 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:19:05.311120 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:19:05.311120 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:19:05.311120 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:19:05.311120 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:19:05.311120 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:19:05.311120 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:19:05.311120 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:19:05.311120 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:19:05.311120 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 17:19:05.311120 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:19:05.311120 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 17:19:05.311120 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:19:05.311120 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 484/975] 10.222.0.102 () {56 vars in 1131 bytes} [Thu Sep 1 17:19:05 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:19:05.312742 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f91d6ba-c5b1-49aa-b3a6-2faa9803cabf None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:05.313210 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1648b7-3f2f-41dc-a9d5-06ed8cc00baa None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:05.402156 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:05.404013 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:05.424642 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1648b7-3f2f-41dc-a9d5-06ed8cc00baa None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:05.427664 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f91d6ba-c5b1-49aa-b3a6-2faa9803cabf None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:05.461367 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4c39ab-374f-49fa-bf0b-c2b724b686ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:05.461984 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4c39ab-374f-49fa-bf0b-c2b724b686ce None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:05.462267 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4c39ab-374f-49fa-bf0b-c2b724b686ce None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:05.462542 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4c39ab-374f-49fa-bf0b-c2b724b686ce None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:05.463168 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4c39ab-374f-49fa-bf0b-c2b724b686ce None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:19:05.463475 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4c39ab-374f-49fa-bf0b-c2b724b686ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:05.464306 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d6f319-6c94-4c1d-afbe-c5d6760f0bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:05.464850 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4c39ab-374f-49fa-bf0b-c2b724b686ce None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:05.464959 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d6f319-6c94-4c1d-afbe-c5d6760f0bcc None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:05.465236 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d6f319-6c94-4c1d-afbe-c5d6760f0bcc None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:05.465540 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d6f319-6c94-4c1d-afbe-c5d6760f0bcc None admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:05.466298 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d6f319-6c94-4c1d-afbe-c5d6760f0bcc None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:19:05.466808 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d6f319-6c94-4c1d-afbe-c5d6760f0bcc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:05.468006 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d6f319-6c94-4c1d-afbe-c5d6760f0bcc None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:05.475907 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 485/976] 10.222.0.102 () {56 vars in 1127 bytes} [Thu Sep 1 17:19:05 2022] GET /identity/v3/services?name=object-store => generated 130 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:19:05.475907 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07fe1b3-8f67-41d4-b3a2-b48a81f62d15 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:05.475907 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07fe1b3-8f67-41d4-b3a2-b48a81f62d15 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:05.475907 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07fe1b3-8f67-41d4-b3a2-b48a81f62d15 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:05.476588 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 486/977] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:19:05 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Sep 01 17:19:05.477677 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07fe1b3-8f67-41d4-b3a2-b48a81f62d15 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:05.493363 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 492/978] 10.222.0.102 () {56 vars in 1154 bytes} [Thu Sep 1 17:19:05 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 386 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:19:05.493363 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c13d98-3dd9-40a9-91fd-153ad82a1086 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:05.493363 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c13d98-3dd9-40a9-91fd-153ad82a1086 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:05.493363 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c13d98-3dd9-40a9-91fd-153ad82a1086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:05.510812 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-14c13d98-3dd9-40a9-91fd-153ad82a1086 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:19:05.512297 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-14c13d98-3dd9-40a9-91fd-153ad82a1086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:19:05.581046 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07fe1b3-8f67-41d4-b3a2-b48a81f62d15 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:05.581569 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 493/979] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:19:05 2022] POST /identity/v3/auth/tokens => generated 1476 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:19:05.582431 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c13d98-3dd9-40a9-91fd-153ad82a1086 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:05.617715 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc166b1-cb42-428b-9eb9-2747e7fbdd3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:05.618341 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc166b1-cb42-428b-9eb9-2747e7fbdd3e None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:05.618622 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc166b1-cb42-428b-9eb9-2747e7fbdd3e None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:05.618896 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc166b1-cb42-428b-9eb9-2747e7fbdd3e None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:05.619537 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc166b1-cb42-428b-9eb9-2747e7fbdd3e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:19:05.619847 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc166b1-cb42-428b-9eb9-2747e7fbdd3e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:05.621368 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc166b1-cb42-428b-9eb9-2747e7fbdd3e None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:05.631610 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 487/980] 10.222.0.102 () {56 vars in 1127 bytes} [Thu Sep 1 17:19:05 2022] GET /identity/v3/services?type=object-store => generated 358 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:19:05.633021 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc166b1-cb42-428b-9eb9-2747e7fbdd3e None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:05.650367 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:05.667619 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:05.712524 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c13d98-3dd9-40a9-91fd-153ad82a1086 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:05.748568 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adb2193-4af6-4f56-bfa0-37b22683fce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:05.749247 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adb2193-4af6-4f56-bfa0-37b22683fce9 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:05.749247 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adb2193-4af6-4f56-bfa0-37b22683fce9 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:05.749505 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adb2193-4af6-4f56-bfa0-37b22683fce9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:05.755695 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc166b1-cb42-428b-9eb9-2747e7fbdd3e None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:05.790060 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3721de0e-8d5f-4ce0-9c08-7c301528e58e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:05.790660 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3721de0e-8d5f-4ce0-9c08-7c301528e58e None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:05.790939 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3721de0e-8d5f-4ce0-9c08-7c301528e58e None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:05.791326 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3721de0e-8d5f-4ce0-9c08-7c301528e58e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:05.792031 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3721de0e-8d5f-4ce0-9c08-7c301528e58e None admin] RBAC: Adding query filter params (interface=admin, service_id=2cfefc081df04b67a52fd276f3c8eb5e, region_id=RegionOne) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:19:05.792340 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3721de0e-8d5f-4ce0-9c08-7c301528e58e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:05.793798 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3721de0e-8d5f-4ce0-9c08-7c301528e58e None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:05.803877 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 488/981] 10.222.0.102 () {56 vars in 1254 bytes} [Thu Sep 1 17:19:05 2022] GET /identity/v3/endpoints?service_id=2cfefc081df04b67a52fd276f3c8eb5e&interface=admin®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:19:05.805169 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3721de0e-8d5f-4ce0-9c08-7c301528e58e None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:05.839423 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adb2193-4af6-4f56-bfa0-37b22683fce9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:05.841543 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adb2193-4af6-4f56-bfa0-37b22683fce9 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:05.895822 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:05.920301 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:05.926160 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3721de0e-8d5f-4ce0-9c08-7c301528e58e None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:05.957529 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 494/982] 10.222.0.102 () {58 vars in 1287 bytes} [Thu Sep 1 17:19:05 2022] GET /identity/v3/auth/tokens => generated 1476 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 17:19:05.958340 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adb2193-4af6-4f56-bfa0-37b22683fce9 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:05.965196 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c188a88a-6fca-45f8-9d9c-5ec7538625f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:05.965850 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c188a88a-6fca-45f8-9d9c-5ec7538625f0 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:05.966137 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c188a88a-6fca-45f8-9d9c-5ec7538625f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:05.966824 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c188a88a-6fca-45f8-9d9c-5ec7538625f0 None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:05.967439 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c188a88a-6fca-45f8-9d9c-5ec7538625f0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:05.969185 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c188a88a-6fca-45f8-9d9c-5ec7538625f0 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:05.984925 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 489/983] 10.222.0.102 () {56 vars in 1092 bytes} [Thu Sep 1 17:19:05 2022] GET /identity/v3/services => generated 1504 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 17:19:05.984925 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c188a88a-6fca-45f8-9d9c-5ec7538625f0 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:06.070966 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adb2193-4af6-4f56-bfa0-37b22683fce9 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:06.109115 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c188a88a-6fca-45f8-9d9c-5ec7538625f0 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:06.110852 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1eb113-fb1c-4327-99e4-91debf616ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:06.111290 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1eb113-fb1c-4327-99e4-91debf616ce4 None admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:06.112072 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1eb113-fb1c-4327-99e4-91debf616ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:06.112344 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1eb113-fb1c-4327-99e4-91debf616ce4 None admin] PATH_INFO: `/v3/services` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:06.113000 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1eb113-fb1c-4327-99e4-91debf616ce4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:06.115133 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1eb113-fb1c-4327-99e4-91debf616ce4 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:06.132879 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 495/984] 10.222.0.102 () {60 vars in 1146 bytes} [Thu Sep 1 17:19:05 2022] POST /identity/v3/services => generated 243 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:19:06.165195 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:06.171676 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651f4dae-da4a-4821-9fa5-cbb71953ea71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:06.172241 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651f4dae-da4a-4821-9fa5-cbb71953ea71 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:06.172520 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651f4dae-da4a-4821-9fa5-cbb71953ea71 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:06.172793 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651f4dae-da4a-4821-9fa5-cbb71953ea71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:06.185821 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651f4dae-da4a-4821-9fa5-cbb71953ea71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:06.187373 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651f4dae-da4a-4821-9fa5-cbb71953ea71 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:06.204455 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-651f4dae-da4a-4821-9fa5-cbb71953ea71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:19:06.204455 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:19:06.204455 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:19:06.204455 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:19:06.204455 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:19:06.204455 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:19:06.204455 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:19:06.204455 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:19:06.204455 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:19:06.204455 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:19:06.204455 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:19:06.204455 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:19:06.204455 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:19:06.204455 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:19:06.204455 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:19:06.204455 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:19:06.205825 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:19:06.205825 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:19:06.205825 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:19:06.205825 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:19:06.205825 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:19:06.205825 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:19:06.205825 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:19:06.205825 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:19:06.205825 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:19:06.205825 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:19:06.205825 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:19:06.205825 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:19:06.205825 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:19:06.205825 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:19:06.205825 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:19:06.205825 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:19:06.205825 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:19:06.207128 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:19:06.207128 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:19:06.207128 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:19:06.207128 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 490/985] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:19:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:19:06.211564 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1eb113-fb1c-4327-99e4-91debf616ce4 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:06.244073 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:06.327929 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1eb113-fb1c-4327-99e4-91debf616ce4 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:06.364262 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaf3387-203d-4455-9db5-db79c1607dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:06.364876 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaf3387-203d-4455-9db5-db79c1607dcf None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:06.365158 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaf3387-203d-4455-9db5-db79c1607dcf None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:06.365468 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaf3387-203d-4455-9db5-db79c1607dcf None admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:06.366160 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaf3387-203d-4455-9db5-db79c1607dcf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:19:06.366472 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaf3387-203d-4455-9db5-db79c1607dcf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:06.367846 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaf3387-203d-4455-9db5-db79c1607dcf None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:06.385030 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 496/986] 10.222.0.102 () {56 vars in 1114 bytes} [Thu Sep 1 17:19:06 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:19:06.386708 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651f4dae-da4a-4821-9fa5-cbb71953ea71 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:06.495339 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651f4dae-da4a-4821-9fa5-cbb71953ea71 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:06.530409 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8b6763-fdbc-45e8-9c6e-179ddf7b9327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:06.531044 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8b6763-fdbc-45e8-9c6e-179ddf7b9327 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:06.531326 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8b6763-fdbc-45e8-9c6e-179ddf7b9327 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:06.531600 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8b6763-fdbc-45e8-9c6e-179ddf7b9327 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:06.539143 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8b6763-fdbc-45e8-9c6e-179ddf7b9327 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:06.540640 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8b6763-fdbc-45e8-9c6e-179ddf7b9327 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:06.552748 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-0e8b6763-fdbc-45e8-9c6e-179ddf7b9327 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:19:06.552748 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:19:06.552748 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:19:06.552748 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:19:06.552748 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:19:06.552748 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:19:06.552748 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:19:06.552748 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:19:06.552748 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:19:06.552748 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:19:06.552748 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:19:06.552748 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:19:06.552748 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:19:06.552748 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:19:06.552748 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:19:06.553945 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:19:06.553945 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:19:06.553945 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:19:06.553945 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:19:06.553945 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:19:06.553945 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:19:06.553945 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:19:06.553945 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:19:06.553945 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:19:06.553945 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:19:06.553945 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:19:06.553945 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:19:06.553945 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:19:06.553945 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:19:06.553945 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:19:06.553945 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:19:06.553945 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:19:06.554806 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:19:06.554806 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:19:06.554806 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:19:06.554806 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:19:06.554806 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:19:06.554806 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:19:06.554806 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:19:06.554806 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:19:06.557364 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 491/987] 10.222.0.102 () {56 vars in 1116 bytes} [Thu Sep 1 17:19:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:19:06.559967 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaf3387-203d-4455-9db5-db79c1607dcf None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:06.650352 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:06.666491 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaf3387-203d-4455-9db5-db79c1607dcf None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:06.700797 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cec9874-d711-4cd9-bfe8-7a1adba52597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:06.701421 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cec9874-d711-4cd9-bfe8-7a1adba52597 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:06.701704 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cec9874-d711-4cd9-bfe8-7a1adba52597 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:06.701978 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cec9874-d711-4cd9-bfe8-7a1adba52597 None admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:06.702659 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cec9874-d711-4cd9-bfe8-7a1adba52597 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:19:06.702966 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cec9874-d711-4cd9-bfe8-7a1adba52597 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:06.704378 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cec9874-d711-4cd9-bfe8-7a1adba52597 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:06.716595 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 497/988] 10.222.0.102 () {56 vars in 1153 bytes} [Thu Sep 1 17:19:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:19:06.722882 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8b6763-fdbc-45e8-9c6e-179ddf7b9327 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:06.772075 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:06.829992 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8b6763-fdbc-45e8-9c6e-179ddf7b9327 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:06.866794 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f8839f-18a1-4a33-baff-d1c36e957186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:06.867407 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f8839f-18a1-4a33-baff-d1c36e957186 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:06.867688 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f8839f-18a1-4a33-baff-d1c36e957186 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:06.867961 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f8839f-18a1-4a33-baff-d1c36e957186 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:06.868679 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f8839f-18a1-4a33-baff-d1c36e957186 None admin] RBAC: Adding query filter params (role.id=9329b00eca174ca5a29094e84e243fa4, scope.project.id=e78fff7f1f6e4f68b8d233ad2e5bbd4f, user.id=575a76b99ea945efb1024fbb3dc81aaf) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:19:06.868986 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f8839f-18a1-4a33-baff-d1c36e957186 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:06.870476 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f8839f-18a1-4a33-baff-d1c36e957186 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:06.881381 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 492/989] 10.222.0.102 () {56 vars in 1379 bytes} [Thu Sep 1 17:19:06 2022] GET /identity/v3/role_assignments?user.id=575a76b99ea945efb1024fbb3dc81aaf&scope.project.id=e78fff7f1f6e4f68b8d233ad2e5bbd4f&role.id=9329b00eca174ca5a29094e84e243fa4 => generated 610 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:19:07.894857 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d4f589-ce69-4a81-8f6b-e487decb9289 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:07.895395 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d4f589-ce69-4a81-8f6b-e487decb9289 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:07.895395 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d4f589-ce69-4a81-8f6b-e487decb9289 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:07.897363 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 498/990] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:19:07 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:19:07.902060 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f836320-72f6-47a4-a133-a4b7b8d042f2 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:07.902461 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f836320-72f6-47a4-a133-a4b7b8d042f2 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:07.902624 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f836320-72f6-47a4-a133-a4b7b8d042f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:07.921416 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-5f836320-72f6-47a4-a133-a4b7b8d042f2 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:19:07.922625 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-5f836320-72f6-47a4-a133-a4b7b8d042f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:19:07.995982 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 493/991] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:19:07 2022] POST /identity/v3/auth/tokens => generated 1572 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:19:08.075880 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eadafb-a99f-42b3-a00b-a8e135ed04d5 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:08.076160 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eadafb-a99f-42b3-a00b-a8e135ed04d5 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:08.076359 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eadafb-a99f-42b3-a00b-a8e135ed04d5 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:08.077894 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 499/992] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:19:08 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:19:08.083594 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba7fb50-691d-4c63-8f9b-c35f381eb1e1 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:08.083893 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba7fb50-691d-4c63-8f9b-c35f381eb1e1 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:08.084167 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba7fb50-691d-4c63-8f9b-c35f381eb1e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:08.102272 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-1ba7fb50-691d-4c63-8f9b-c35f381eb1e1 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:19:08.103710 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-1ba7fb50-691d-4c63-8f9b-c35f381eb1e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:19:08.133947 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:08.176300 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 494/993] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:19:08 2022] POST /identity/v3/auth/tokens => generated 1572 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:19:08.269061 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eadafb-a99f-42b3-a00b-a8e135ed04d5 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:08.373989 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eadafb-a99f-42b3-a00b-a8e135ed04d5 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:08.408780 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f75e79-c281-48da-bee5-1b52e4008aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:08.409450 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f75e79-c281-48da-bee5-1b52e4008aad None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:08.409735 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f75e79-c281-48da-bee5-1b52e4008aad None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:08.410014 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f75e79-c281-48da-bee5-1b52e4008aad None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:08.410596 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f75e79-c281-48da-bee5-1b52e4008aad None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:08.411964 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f75e79-c281-48da-bee5-1b52e4008aad None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:08.421530 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-10f75e79-c281-48da-bee5-1b52e4008aad None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 17:19:08.421530 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:19:08.421530 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 17:19:08.421530 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 17:19:08.421530 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 17:19:08.421530 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 17:19:08.421530 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 17:19:08.421530 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:19:08.421530 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 17:19:08.421530 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:19:08.421530 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 17:19:08.421530 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:19:08.421530 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:19:08.421530 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:19:08.421530 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:19:08.421530 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:19:08.421530 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:19:08.422510 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:19:08.422510 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:19:08.422510 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:19:08.422510 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:19:08.422510 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:19:08.422510 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:19:08.422510 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 17:19:08.422510 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 17:19:08.422510 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 17:19:08.422510 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 17:19:08.422510 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:19:08.422510 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:19:08.422510 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:19:08.422510 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:19:08.422510 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:19:08.422510 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:19:08.423359 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:19:08.423359 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:19:08.423359 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:19:08.423359 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:19:08.423359 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:19:08.423359 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:19:08.423359 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:19:08.423359 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:19:08.423359 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:19:08.423359 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:19:08.423359 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 17:19:08.423359 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:19:08.423359 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 17:19:08.423359 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:19:08.425412 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 500/994] 10.222.0.102 () {56 vars in 1131 bytes} [Thu Sep 1 17:19:08 2022] GET /identity/v3/services/object-store => generated 93 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:19:08.429160 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba7fb50-691d-4c63-8f9b-c35f381eb1e1 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:08.449124 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f75e79-c281-48da-bee5-1b52e4008aad None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:08.462336 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:08.540929 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba7fb50-691d-4c63-8f9b-c35f381eb1e1 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:08.556652 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f75e79-c281-48da-bee5-1b52e4008aad None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:08.578826 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272c9de5-ac30-4038-bfd2-69032c9cedbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:08.579589 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272c9de5-ac30-4038-bfd2-69032c9cedbf None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:08.579873 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272c9de5-ac30-4038-bfd2-69032c9cedbf None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:08.580176 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272c9de5-ac30-4038-bfd2-69032c9cedbf None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:08.580808 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272c9de5-ac30-4038-bfd2-69032c9cedbf None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:19:08.581112 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272c9de5-ac30-4038-bfd2-69032c9cedbf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:08.582585 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272c9de5-ac30-4038-bfd2-69032c9cedbf None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:08.592195 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 495/995] 10.222.0.102 () {56 vars in 1127 bytes} [Thu Sep 1 17:19:08 2022] GET /identity/v3/services?name=object-store => generated 130 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:19:08.593040 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d720bc47-0726-4d19-85f9-65e65d6e6731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:08.593246 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272c9de5-ac30-4038-bfd2-69032c9cedbf None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:08.594155 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d720bc47-0726-4d19-85f9-65e65d6e6731 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:08.594434 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d720bc47-0726-4d19-85f9-65e65d6e6731 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:08.594705 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d720bc47-0726-4d19-85f9-65e65d6e6731 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:08.595319 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d720bc47-0726-4d19-85f9-65e65d6e6731 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:08.596703 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d720bc47-0726-4d19-85f9-65e65d6e6731 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:08.605796 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-d720bc47-0726-4d19-85f9-65e65d6e6731 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 17:19:08.605796 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:19:08.605796 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 17:19:08.605796 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 17:19:08.605796 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 17:19:08.605796 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 17:19:08.605796 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 17:19:08.605796 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:19:08.605796 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 17:19:08.605796 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:19:08.605796 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 17:19:08.605796 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:19:08.605796 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:19:08.605796 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:19:08.605796 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:19:08.605796 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:19:08.605796 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:19:08.606830 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:19:08.606830 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:19:08.606830 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:19:08.606830 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:19:08.606830 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:19:08.606830 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:19:08.606830 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 17:19:08.606830 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 17:19:08.606830 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 17:19:08.606830 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 17:19:08.606830 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:19:08.606830 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:19:08.606830 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:19:08.606830 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:19:08.606830 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:19:08.606830 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:19:08.607676 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:19:08.607676 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:19:08.607676 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:19:08.607676 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:19:08.607676 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:19:08.607676 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:19:08.607676 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:19:08.607676 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:19:08.607676 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:19:08.607676 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:19:08.607676 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 17:19:08.607676 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:19:08.607676 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 17:19:08.607676 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:19:08.607676 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 501/996] 10.222.0.102 () {56 vars in 1116 bytes} [Thu Sep 1 17:19:08 2022] GET /identity/v3/services/compute => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:19:08.613216 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d720bc47-0726-4d19-85f9-65e65d6e6731 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:08.631346 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:08.696765 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:08.706477 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272c9de5-ac30-4038-bfd2-69032c9cedbf None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:08.734100 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d720bc47-0726-4d19-85f9-65e65d6e6731 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:08.749504 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4555443-4595-4611-987a-5766617199fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:08.750051 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4555443-4595-4611-987a-5766617199fe None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:08.750333 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4555443-4595-4611-987a-5766617199fe None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:08.750719 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4555443-4595-4611-987a-5766617199fe None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:08.751636 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4555443-4595-4611-987a-5766617199fe None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:19:08.751746 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4555443-4595-4611-987a-5766617199fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:08.753175 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4555443-4595-4611-987a-5766617199fe None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:08.758480 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 496/997] 10.222.0.102 () {56 vars in 1127 bytes} [Thu Sep 1 17:19:08 2022] GET /identity/v3/services?type=object-store => generated 358 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:19:08.764196 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4555443-4595-4611-987a-5766617199fe None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:08.769830 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe2014e-824b-4c6a-a980-bdcccfd4d409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:08.770447 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe2014e-824b-4c6a-a980-bdcccfd4d409 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:08.770729 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe2014e-824b-4c6a-a980-bdcccfd4d409 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:08.771006 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe2014e-824b-4c6a-a980-bdcccfd4d409 None admin] PATH_INFO: `/v3/services` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:08.771727 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe2014e-824b-4c6a-a980-bdcccfd4d409 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:19:08.772116 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe2014e-824b-4c6a-a980-bdcccfd4d409 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:08.773997 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe2014e-824b-4c6a-a980-bdcccfd4d409 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:08.780026 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 502/998] 10.222.0.102 () {56 vars in 1117 bytes} [Thu Sep 1 17:19:08 2022] GET /identity/v3/services?name=compute => generated 125 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:19:08.785886 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe2014e-824b-4c6a-a980-bdcccfd4d409 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:08.862671 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:08.869225 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4555443-4595-4611-987a-5766617199fe None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:08.890954 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe2014e-824b-4c6a-a980-bdcccfd4d409 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:08.904196 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ec8d20-805f-4166-8dd6-c0fe3b73936b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:08.904808 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ec8d20-805f-4166-8dd6-c0fe3b73936b None admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:08.905087 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ec8d20-805f-4166-8dd6-c0fe3b73936b None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:08.905387 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ec8d20-805f-4166-8dd6-c0fe3b73936b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:08.906050 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ec8d20-805f-4166-8dd6-c0fe3b73936b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:08.907423 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ec8d20-805f-4166-8dd6-c0fe3b73936b None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:08.923275 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 497/999] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:19:08 2022] POST /identity/v3/endpoints => generated 327 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:19:08.924501 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d38905e-dde0-4906-8c22-51d43ac220ed None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:08.924787 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d38905e-dde0-4906-8c22-51d43ac220ed None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:08.925062 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d38905e-dde0-4906-8c22-51d43ac220ed None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:08.926306 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 498/1000] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:19:08 2022] GET /identity => generated 271 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:19:08.932101 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3fed2a-9f03-4aed-81b0-3ce320718f1b None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:08.932465 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3fed2a-9f03-4aed-81b0-3ce320718f1b None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:08.932945 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3fed2a-9f03-4aed-81b0-3ce320718f1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:08.933895 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fd48f5-1255-4a43-b5c1-cd3a45b09425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:08.934521 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fd48f5-1255-4a43-b5c1-cd3a45b09425 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:08.934801 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fd48f5-1255-4a43-b5c1-cd3a45b09425 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:08.935075 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fd48f5-1255-4a43-b5c1-cd3a45b09425 None admin] PATH_INFO: `/v3/services` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:08.935723 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fd48f5-1255-4a43-b5c1-cd3a45b09425 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:19:08.936044 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fd48f5-1255-4a43-b5c1-cd3a45b09425 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:08.937478 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fd48f5-1255-4a43-b5c1-cd3a45b09425 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:08.940016 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:08.945752 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 503/1001] 10.222.0.102 () {56 vars in 1117 bytes} [Thu Sep 1 17:19:08 2022] GET /identity/v3/services?type=compute => generated 354 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:19:08.951440 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fd48f5-1255-4a43-b5c1-cd3a45b09425 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:08.957807 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-ad3fed2a-9f03-4aed-81b0-3ce320718f1b None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:19:08.959407 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-ad3fed2a-9f03-4aed-81b0-3ce320718f1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:19:09.043078 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 499/1002] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:19:08 2022] POST /identity/v3/auth/tokens => generated 1720 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 17:19:09.071506 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fd48f5-1255-4a43-b5c1-cd3a45b09425 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:09.106365 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6a793e-9e14-44f9-9fcf-6ccc5750ce1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:09.106976 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6a793e-9e14-44f9-9fcf-6ccc5750ce1d None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:09.107260 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6a793e-9e14-44f9-9fcf-6ccc5750ce1d None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:09.107533 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6a793e-9e14-44f9-9fcf-6ccc5750ce1d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:09.108238 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6a793e-9e14-44f9-9fcf-6ccc5750ce1d None admin] RBAC: Adding query filter params (interface=public, service_id=e1720fa4fef8403fbdcecea03f368104, region_id=RegionOne) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:19:09.108545 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6a793e-9e14-44f9-9fcf-6ccc5750ce1d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:09.109935 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6a793e-9e14-44f9-9fcf-6ccc5750ce1d None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:09.115685 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 504/1003] 10.222.0.102 () {56 vars in 1256 bytes} [Thu Sep 1 17:19:08 2022] GET /identity/v3/endpoints?service_id=e1720fa4fef8403fbdcecea03f368104&interface=public®ion_id=RegionOne => generated 195 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:19:09.121362 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3fed2a-9f03-4aed-81b0-3ce320718f1b None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:09.210591 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:09.227742 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3fed2a-9f03-4aed-81b0-3ce320718f1b None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:09.263312 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929306f0-7de2-4eae-b352-04191cb6ae4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:09.263933 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929306f0-7de2-4eae-b352-04191cb6ae4f None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:09.264264 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929306f0-7de2-4eae-b352-04191cb6ae4f None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:09.264545 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929306f0-7de2-4eae-b352-04191cb6ae4f None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:09.265156 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929306f0-7de2-4eae-b352-04191cb6ae4f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:09.266616 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929306f0-7de2-4eae-b352-04191cb6ae4f None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:09.273176 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 500/1004] 10.222.0.102 () {56 vars in 1092 bytes} [Thu Sep 1 17:19:09 2022] GET /identity/v3/services => generated 1735 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 17:19:09.311236 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6a793e-9e14-44f9-9fcf-6ccc5750ce1d None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:09.378981 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:09.424721 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6a793e-9e14-44f9-9fcf-6ccc5750ce1d None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:09.458367 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b4d29b-bb93-4039-bed8-8574477a23d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:09.459253 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b4d29b-bb93-4039-bed8-8574477a23d2 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:09.459366 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b4d29b-bb93-4039-bed8-8574477a23d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:09.459605 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b4d29b-bb93-4039-bed8-8574477a23d2 None admin] PATH_INFO: `/v3/services/image` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:09.460184 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b4d29b-bb93-4039-bed8-8574477a23d2 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:09.461620 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b4d29b-bb93-4039-bed8-8574477a23d2 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:09.470336 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-b9b4d29b-bb93-4039-bed8-8574477a23d2 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 17:19:09.470336 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:19:09.470336 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 17:19:09.470336 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 17:19:09.470336 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 17:19:09.470336 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 17:19:09.470336 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 17:19:09.470336 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:19:09.470336 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 17:19:09.470336 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:19:09.470336 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 17:19:09.470336 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:19:09.470336 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:19:09.470336 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:19:09.470336 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:19:09.470336 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:19:09.470336 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:19:09.471848 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:19:09.471848 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:19:09.471848 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:19:09.471848 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:19:09.471848 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:19:09.471848 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:19:09.471848 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 17:19:09.471848 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 17:19:09.471848 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 17:19:09.471848 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 17:19:09.471848 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:19:09.471848 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:19:09.471848 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:19:09.471848 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:19:09.471848 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:19:09.471848 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:19:09.472676 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:19:09.472676 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:19:09.472676 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:19:09.472676 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:19:09.472676 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:19:09.472676 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:19:09.472676 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:19:09.472676 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:19:09.472676 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:19:09.472676 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:19:09.472676 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 17:19:09.472676 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:19:09.472676 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 17:19:09.472676 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:19:09.472676 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 505/1005] 10.222.0.102 () {56 vars in 1110 bytes} [Thu Sep 1 17:19:09 2022] GET /identity/v3/services/image => generated 86 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:19:09.479413 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929306f0-7de2-4eae-b352-04191cb6ae4f None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:09.583926 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929306f0-7de2-4eae-b352-04191cb6ae4f None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:09.621511 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053decff-cc31-409c-b69b-7e47bd137160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:09.622118 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053decff-cc31-409c-b69b-7e47bd137160 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:09.622398 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053decff-cc31-409c-b69b-7e47bd137160 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:09.622671 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053decff-cc31-409c-b69b-7e47bd137160 None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:09.623294 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053decff-cc31-409c-b69b-7e47bd137160 None admin] RBAC: Adding query filter params (name=image) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:19:09.623597 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053decff-cc31-409c-b69b-7e47bd137160 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:09.624969 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053decff-cc31-409c-b69b-7e47bd137160 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:09.630113 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 501/1006] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:19:09 2022] GET /identity/v3/services?name=image => generated 123 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:19:09.635599 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b4d29b-bb93-4039-bed8-8574477a23d2 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:09.734722 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b4d29b-bb93-4039-bed8-8574477a23d2 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:09.761746 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:09.771760 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77b1a36-9dc4-4bb8-aba2-e0ffa0f28428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:09.772392 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77b1a36-9dc4-4bb8-aba2-e0ffa0f28428 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:09.772674 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77b1a36-9dc4-4bb8-aba2-e0ffa0f28428 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:09.772946 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77b1a36-9dc4-4bb8-aba2-e0ffa0f28428 None admin] PATH_INFO: `/v3/services` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:09.773598 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77b1a36-9dc4-4bb8-aba2-e0ffa0f28428 None admin] RBAC: Adding query filter params (type=image) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:19:09.773904 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77b1a36-9dc4-4bb8-aba2-e0ffa0f28428 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:09.775255 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77b1a36-9dc4-4bb8-aba2-e0ffa0f28428 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:09.785555 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 506/1007] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:19:09 2022] GET /identity/v3/services?type=image => generated 123 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:19:10.893919 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57442ca-7f9e-41f7-9c13-984c4324b0b0 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:10.894433 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57442ca-7f9e-41f7-9c13-984c4324b0b0 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:10.894508 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57442ca-7f9e-41f7-9c13-984c4324b0b0 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:10.895852 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 502/1008] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:19:10 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:19:10.901021 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbfab7e-81ef-45f9-8fb3-58e8dc0541b8 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:10.901471 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbfab7e-81ef-45f9-8fb3-58e8dc0541b8 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:10.901659 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbfab7e-81ef-45f9-8fb3-58e8dc0541b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:10.920424 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-5fbfab7e-81ef-45f9-8fb3-58e8dc0541b8 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:19:10.921681 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-5fbfab7e-81ef-45f9-8fb3-58e8dc0541b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:19:10.995432 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 507/1009] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:19:10 2022] POST /identity/v3/auth/tokens => generated 1720 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:19:11.248339 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ef2686-251e-4caf-afe8-3ed5d9424c15 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:11.248639 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ef2686-251e-4caf-afe8-3ed5d9424c15 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:11.248805 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ef2686-251e-4caf-afe8-3ed5d9424c15 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:11.250320 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 503/1010] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:19:11 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:19:11.255467 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eab09bd-7776-47b1-a5cf-bf9a969580bf None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:11.255755 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eab09bd-7776-47b1-a5cf-bf9a969580bf None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:11.256031 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eab09bd-7776-47b1-a5cf-bf9a969580bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:11.264064 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ef2686-251e-4caf-afe8-3ed5d9424c15 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:11.277083 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-0eab09bd-7776-47b1-a5cf-bf9a969580bf None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:19:11.278311 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-0eab09bd-7776-47b1-a5cf-bf9a969580bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:19:11.281326 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:11.359672 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 508/1011] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:19:11 2022] POST /identity/v3/auth/tokens => generated 1720 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:19:11.379670 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ef2686-251e-4caf-afe8-3ed5d9424c15 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:11.415723 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c29daf-3e29-4b5e-ba62-ea12a0d393ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:11.416355 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c29daf-3e29-4b5e-ba62-ea12a0d393ed None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:11.416633 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c29daf-3e29-4b5e-ba62-ea12a0d393ed None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:11.416905 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c29daf-3e29-4b5e-ba62-ea12a0d393ed None admin] PATH_INFO: `/v3/domains/default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:11.418032 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c29daf-3e29-4b5e-ba62-ea12a0d393ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:11.419494 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c29daf-3e29-4b5e-ba62-ea12a0d393ed None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:11.425352 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 504/1012] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:19:11 2022] GET /identity/v3/domains/default => generated 200 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:19:11.426847 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eab09bd-7776-47b1-a5cf-bf9a969580bf None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:11.481200 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:11.531687 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eab09bd-7776-47b1-a5cf-bf9a969580bf None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:11.565943 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6912c5-dc71-4d7f-804e-e2b9e0233fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:11.566552 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6912c5-dc71-4d7f-804e-e2b9e0233fdd None admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:11.566830 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6912c5-dc71-4d7f-804e-e2b9e0233fdd None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:11.567098 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6912c5-dc71-4d7f-804e-e2b9e0233fdd None admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:11.567745 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6912c5-dc71-4d7f-804e-e2b9e0233fdd None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:11.569543 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6912c5-dc71-4d7f-804e-e2b9e0233fdd None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:11.602679 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 509/1013] 10.222.0.102 () {60 vars in 1146 bytes} [Thu Sep 1 17:19:11 2022] POST /identity/v3/projects => generated 312 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:19:11.633760 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c29daf-3e29-4b5e-ba62-ea12a0d393ed None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:11.725613 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:11.732836 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9dc046-8cc2-42e6-8d25-70298aac691b None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:11.733168 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9dc046-8cc2-42e6-8d25-70298aac691b None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:11.733411 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9dc046-8cc2-42e6-8d25-70298aac691b None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:11.734750 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 510/1014] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:19:11 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Sep 01 17:19:11.741855 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7260b4eb-a55b-4770-bc5f-03256d235d97 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:11.741855 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7260b4eb-a55b-4770-bc5f-03256d235d97 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:11.741855 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7260b4eb-a55b-4770-bc5f-03256d235d97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:11.750272 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c29daf-3e29-4b5e-ba62-ea12a0d393ed None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:11.760784 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-7260b4eb-a55b-4770-bc5f-03256d235d97 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:19:11.762001 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-7260b4eb-a55b-4770-bc5f-03256d235d97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:19:11.788156 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49033c8-875f-4e56-905d-2ba2d8ccb0d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:11.788721 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49033c8-875f-4e56-905d-2ba2d8ccb0d9 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:11.789851 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49033c8-875f-4e56-905d-2ba2d8ccb0d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:11.789851 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49033c8-875f-4e56-905d-2ba2d8ccb0d9 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:11.790022 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49033c8-875f-4e56-905d-2ba2d8ccb0d9 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:11.791400 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49033c8-875f-4e56-905d-2ba2d8ccb0d9 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:11.803063 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-d49033c8-875f-4e56-905d-2ba2d8ccb0d9 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 17:19:11.803063 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:19:11.803063 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 17:19:11.803063 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 17:19:11.803063 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 17:19:11.803063 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 17:19:11.803063 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 17:19:11.803063 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:19:11.803063 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 17:19:11.803063 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:19:11.803063 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 17:19:11.803063 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:19:11.803063 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:19:11.803063 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:19:11.803063 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:19:11.803063 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:19:11.803063 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:19:11.804069 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:19:11.804069 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:19:11.804069 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:19:11.804069 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:19:11.804069 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:19:11.804069 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:19:11.804069 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 17:19:11.804069 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 17:19:11.804069 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 17:19:11.804069 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 17:19:11.804069 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:19:11.804069 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:19:11.804069 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:19:11.804069 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:19:11.804069 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:19:11.804069 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:19:11.805024 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:19:11.805024 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:19:11.805024 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:19:11.805024 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:19:11.805024 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:19:11.805024 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:19:11.805024 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:19:11.805024 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:19:11.805024 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:19:11.805024 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:19:11.805024 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 17:19:11.805024 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:19:11.805024 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 17:19:11.805024 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:19:11.808647 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 505/1015] 10.222.0.102 () {56 vars in 1116 bytes} [Thu Sep 1 17:19:11 2022] GET /identity/v3/services/compute => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:19:11.809927 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49033c8-875f-4e56-905d-2ba2d8ccb0d9 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:11.846919 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 511/1016] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:19:11 2022] POST /identity/v3/auth/tokens => generated 1720 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 17:19:11.921396 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49033c8-875f-4e56-905d-2ba2d8ccb0d9 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:11.955861 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2a2c7d-80c2-44f2-a168-46de9a1519d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:11.956470 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2a2c7d-80c2-44f2-a168-46de9a1519d2 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:11.956749 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2a2c7d-80c2-44f2-a168-46de9a1519d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:11.957023 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2a2c7d-80c2-44f2-a168-46de9a1519d2 None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:11.957677 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2a2c7d-80c2-44f2-a168-46de9a1519d2 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:19:11.957981 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2a2c7d-80c2-44f2-a168-46de9a1519d2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:11.959385 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2a2c7d-80c2-44f2-a168-46de9a1519d2 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:11.964258 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 506/1017] 10.222.0.102 () {56 vars in 1117 bytes} [Thu Sep 1 17:19:11 2022] GET /identity/v3/services?name=compute => generated 125 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:19:11.969968 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7260b4eb-a55b-4770-bc5f-03256d235d97 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:12.001879 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:12.076132 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7260b4eb-a55b-4770-bc5f-03256d235d97 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:12.112835 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6626f149-60f5-42f7-9814-63bedb121ddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:12.113157 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2a2c7d-80c2-44f2-a168-46de9a1519d2 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:12.113476 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6626f149-60f5-42f7-9814-63bedb121ddb None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:12.113757 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6626f149-60f5-42f7-9814-63bedb121ddb None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:12.114030 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6626f149-60f5-42f7-9814-63bedb121ddb None admin] PATH_INFO: `/v3/services` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:12.114659 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6626f149-60f5-42f7-9814-63bedb121ddb None admin] RBAC: Adding query filter params (type=compute) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:19:12.114964 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6626f149-60f5-42f7-9814-63bedb121ddb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:12.116320 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6626f149-60f5-42f7-9814-63bedb121ddb None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:12.125397 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 512/1018] 10.222.0.102 () {56 vars in 1117 bytes} [Thu Sep 1 17:19:11 2022] GET /identity/v3/services?type=compute => generated 354 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:19:12.128593 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6626f149-60f5-42f7-9814-63bedb121ddb None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:12.130730 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:12.224084 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2a2c7d-80c2-44f2-a168-46de9a1519d2 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:12.233551 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:12.241095 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6626f149-60f5-42f7-9814-63bedb121ddb None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:12.260093 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16182920-78e8-47c0-8a7e-6be03258802f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:12.261221 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16182920-78e8-47c0-8a7e-6be03258802f None admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:12.261554 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16182920-78e8-47c0-8a7e-6be03258802f None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:12.261827 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16182920-78e8-47c0-8a7e-6be03258802f None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:12.262492 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16182920-78e8-47c0-8a7e-6be03258802f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:12.263862 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16182920-78e8-47c0-8a7e-6be03258802f None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:12.280132 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d20c2c-35f0-4f61-b130-7d307658fd25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:12.280738 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d20c2c-35f0-4f61-b130-7d307658fd25 None admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:12.281020 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d20c2c-35f0-4f61-b130-7d307658fd25 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:12.285347 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d20c2c-35f0-4f61-b130-7d307658fd25 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:12.285497 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 507/1019] 10.222.0.102 () {60 vars in 1146 bytes} [Thu Sep 1 17:19:12 2022] POST /identity/v3/services => generated 243 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:19:12.285577 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d20c2c-35f0-4f61-b130-7d307658fd25 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:12.285577 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d20c2c-35f0-4f61-b130-7d307658fd25 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:12.309397 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 513/1020] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:19:12 2022] POST /identity/v3/endpoints => generated 336 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:19:13.549914 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67608f2-c5cd-4347-9d42-a531f19d0d58 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:13.550459 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67608f2-c5cd-4347-9d42-a531f19d0d58 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:13.550459 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67608f2-c5cd-4347-9d42-a531f19d0d58 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:13.551965 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 508/1021] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:19:13 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:19:13.557136 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d051178-0c3b-4a86-a9fa-8145e0d29df6 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:13.557579 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d051178-0c3b-4a86-a9fa-8145e0d29df6 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:13.557763 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d051178-0c3b-4a86-a9fa-8145e0d29df6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:13.576536 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-0d051178-0c3b-4a86-a9fa-8145e0d29df6 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:19:13.577732 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-0d051178-0c3b-4a86-a9fa-8145e0d29df6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:19:13.649654 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 514/1022] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:19:13 2022] POST /identity/v3/auth/tokens => generated 1973 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:19:13.919866 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67608f2-c5cd-4347-9d42-a531f19d0d58 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:13.998659 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:14.036377 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67608f2-c5cd-4347-9d42-a531f19d0d58 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:14.072078 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72581db-a3c0-499e-970a-bd7b713b6a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:14.072626 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72581db-a3c0-499e-970a-bd7b713b6a5f None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:14.072902 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72581db-a3c0-499e-970a-bd7b713b6a5f None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:14.073204 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72581db-a3c0-499e-970a-bd7b713b6a5f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:14.074319 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72581db-a3c0-499e-970a-bd7b713b6a5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:14.075744 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72581db-a3c0-499e-970a-bd7b713b6a5f None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:14.077578 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 509/1023] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:19:13 2022] GET /identity/v3/domains/default => generated 200 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:19:14.082982 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d051178-0c3b-4a86-a9fa-8145e0d29df6 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:14.159907 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:14.183909 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d051178-0c3b-4a86-a9fa-8145e0d29df6 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:14.219719 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378a1aa6-a715-4beb-bba1-9ea29ce3cec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:14.221126 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378a1aa6-a715-4beb-bba1-9ea29ce3cec6 None admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:14.221126 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378a1aa6-a715-4beb-bba1-9ea29ce3cec6 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:14.221447 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378a1aa6-a715-4beb-bba1-9ea29ce3cec6 None admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:14.222084 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378a1aa6-a715-4beb-bba1-9ea29ce3cec6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:14.223514 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378a1aa6-a715-4beb-bba1-9ea29ce3cec6 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:14.226554 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1c4d3c-1c01-4ffb-930f-0552e37a2f3e None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:14.226846 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1c4d3c-1c01-4ffb-930f-0552e37a2f3e None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:14.227124 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1c4d3c-1c01-4ffb-930f-0552e37a2f3e None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:14.228889 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 510/1024] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:19:14 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:19:14.233578 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8080d031-89d7-4b58-99b4-36684ee5480d None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:14.233868 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8080d031-89d7-4b58-99b4-36684ee5480d None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:14.234143 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8080d031-89d7-4b58-99b4-36684ee5480d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:14.253987 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-8080d031-89d7-4b58-99b4-36684ee5480d None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:19:14.255224 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-8080d031-89d7-4b58-99b4-36684ee5480d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:19:14.257441 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 515/1025] 10.222.0.102 () {60 vars in 1137 bytes} [Thu Sep 1 17:19:14 2022] POST /identity/v3/users => generated 286 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:19:14.333332 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 511/1026] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:19:14 2022] POST /identity/v3/auth/tokens => generated 1973 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 17:19:14.368104 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae7bb14-9f89-4b24-9299-a0265eee9a88 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:14.368426 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae7bb14-9f89-4b24-9299-a0265eee9a88 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:14.368735 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae7bb14-9f89-4b24-9299-a0265eee9a88 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:14.370306 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 516/1027] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:19:14 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:19:14.375991 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03d0374-9f5e-4306-a630-f8ea46d33875 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:14.376398 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03d0374-9f5e-4306-a630-f8ea46d33875 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:14.376837 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03d0374-9f5e-4306-a630-f8ea46d33875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:14.398087 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-c03d0374-9f5e-4306-a630-f8ea46d33875 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:19:14.398866 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-c03d0374-9f5e-4306-a630-f8ea46d33875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:19:14.468826 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:14.475404 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 512/1028] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:19:14 2022] POST /identity/v3/auth/tokens => generated 1973 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:19:14.613157 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae7bb14-9f89-4b24-9299-a0265eee9a88 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:14.720432 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae7bb14-9f89-4b24-9299-a0265eee9a88 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:14.750731 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03d0374-9f5e-4306-a630-f8ea46d33875 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:14.758694 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d681c0-de31-4874-a5a8-9610d32a721b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:14.759508 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d681c0-de31-4874-a5a8-9610d32a721b None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:14.759967 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d681c0-de31-4874-a5a8-9610d32a721b None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:14.760461 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d681c0-de31-4874-a5a8-9610d32a721b None admin] PATH_INFO: `/v3/services/image` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:14.761212 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d681c0-de31-4874-a5a8-9610d32a721b None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:14.762748 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d681c0-de31-4874-a5a8-9610d32a721b None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:14.765536 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:14.775557 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-34d681c0-de31-4874-a5a8-9610d32a721b None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 17:19:14.775557 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:19:14.775557 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 17:19:14.775557 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 17:19:14.775557 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 17:19:14.775557 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 17:19:14.775557 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 17:19:14.775557 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:19:14.775557 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 17:19:14.775557 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:19:14.775557 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 17:19:14.775557 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:19:14.775557 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:19:14.775557 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:19:14.775557 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:19:14.775557 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:19:14.775557 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:19:14.776732 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:19:14.776732 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:19:14.776732 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:19:14.776732 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:19:14.776732 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:19:14.776732 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:19:14.776732 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 17:19:14.776732 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 17:19:14.776732 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 17:19:14.776732 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 17:19:14.776732 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:19:14.776732 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:19:14.776732 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:19:14.776732 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:19:14.776732 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:19:14.776732 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:19:14.777789 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:19:14.777789 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:19:14.777789 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:19:14.777789 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:19:14.777789 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:19:14.777789 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:19:14.777789 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:19:14.777789 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:19:14.777789 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:19:14.777789 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:19:14.777789 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 17:19:14.777789 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:19:14.777789 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 17:19:14.777789 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:19:14.780337 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 517/1029] 10.222.0.102 () {56 vars in 1110 bytes} [Thu Sep 1 17:19:14 2022] GET /identity/v3/services/image => generated 86 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:19:14.786763 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d681c0-de31-4874-a5a8-9610d32a721b None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:14.868750 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03d0374-9f5e-4306-a630-f8ea46d33875 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:14.899503 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d681c0-de31-4874-a5a8-9610d32a721b None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:14.904857 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a11267-8354-44c4-88d3-7b486cc975e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:14.905526 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a11267-8354-44c4-88d3-7b486cc975e5 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:14.905808 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a11267-8354-44c4-88d3-7b486cc975e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:14.906081 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a11267-8354-44c4-88d3-7b486cc975e5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:14.917268 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a11267-8354-44c4-88d3-7b486cc975e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:19:14.923054 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-a9a11267-8354-44c4-88d3-7b486cc975e5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 17:19:14.923054 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:19:14.923054 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:19:14.923054 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:19:14.923054 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:19:14.923054 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:19:14.923054 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:19:14.923054 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:19:14.923054 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:19:14.923054 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:19:14.923054 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:19:14.923054 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:19:14.923054 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:19:14.923054 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:19:14.923054 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:19:14.923054 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:19:14.924786 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:19:14.924786 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:19:14.924786 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:19:14.924786 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:19:14.924786 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:19:14.924786 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:19:14.924786 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:19:14.924786 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:19:14.924786 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:19:14.924786 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:19:14.924786 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:19:14.924786 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:19:14.924786 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:19:14.924786 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:19:14.924786 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:19:14.924786 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:19:14.925855 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:19:14.925855 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:19:14.925855 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:19:14.925855 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:19:14.925855 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:19:14.925855 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:19:14.925855 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:19:14.925855 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:19:14.925855 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:19:14.925855 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:19:14.925855 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 17:19:14.925855 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:19:14.925855 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 513/1030] 10.222.0.102 () {56 vars in 1125 bytes} [Thu Sep 1 17:19:14 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:19:14.930563 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a11267-8354-44c4-88d3-7b486cc975e5 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:14.942647 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e2fad5-270b-43e4-8bd7-1b2e05f384d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:14.943740 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e2fad5-270b-43e4-8bd7-1b2e05f384d0 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:14.944186 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e2fad5-270b-43e4-8bd7-1b2e05f384d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:14.944468 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e2fad5-270b-43e4-8bd7-1b2e05f384d0 None admin] PATH_INFO: `/v3/services` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:14.945109 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e2fad5-270b-43e4-8bd7-1b2e05f384d0 None admin] RBAC: Adding query filter params (name=image) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:19:14.945440 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e2fad5-270b-43e4-8bd7-1b2e05f384d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:14.946811 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e2fad5-270b-43e4-8bd7-1b2e05f384d0 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:14.952234 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 518/1031] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:19:14 2022] GET /identity/v3/services?name=image => generated 123 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:19:14.958639 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e2fad5-270b-43e4-8bd7-1b2e05f384d0 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:14.998058 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:15.035768 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:15.046316 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a11267-8354-44c4-88d3-7b486cc975e5 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:15.078586 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e2fad5-270b-43e4-8bd7-1b2e05f384d0 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:15.083498 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe092fa-f401-4291-9496-0448af9aedf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:15.084105 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe092fa-f401-4291-9496-0448af9aedf0 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:15.084384 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe092fa-f401-4291-9496-0448af9aedf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:15.084657 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe092fa-f401-4291-9496-0448af9aedf0 None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:15.085330 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe092fa-f401-4291-9496-0448af9aedf0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:19:15.085645 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe092fa-f401-4291-9496-0448af9aedf0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:15.087021 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe092fa-f401-4291-9496-0448af9aedf0 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:15.096612 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 514/1032] 10.222.0.102 () {56 vars in 1120 bytes} [Thu Sep 1 17:19:14 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 340 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:19:15.102148 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe092fa-f401-4291-9496-0448af9aedf0 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:15.114835 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbef063-d96f-469c-9b7d-7bebb52a3ac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:15.115439 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbef063-d96f-469c-9b7d-7bebb52a3ac3 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:15.115723 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbef063-d96f-469c-9b7d-7bebb52a3ac3 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:15.116001 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbef063-d96f-469c-9b7d-7bebb52a3ac3 None admin] PATH_INFO: `/v3/services` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:15.116624 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbef063-d96f-469c-9b7d-7bebb52a3ac3 None admin] RBAC: Adding query filter params (type=image) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:19:15.116930 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbef063-d96f-469c-9b7d-7bebb52a3ac3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:15.118317 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbef063-d96f-469c-9b7d-7bebb52a3ac3 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:15.123268 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 519/1033] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:19:14 2022] GET /identity/v3/services?type=image => generated 352 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:19:15.128993 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbef063-d96f-469c-9b7d-7bebb52a3ac3 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:15.207443 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe092fa-f401-4291-9496-0448af9aedf0 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:15.228676 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:15.237956 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbef063-d96f-469c-9b7d-7bebb52a3ac3 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:15.245625 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe61501-7615-41b8-abcd-d4b7cfe63183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:15.246252 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe61501-7615-41b8-abcd-d4b7cfe63183 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:15.246527 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe61501-7615-41b8-abcd-d4b7cfe63183 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:15.246793 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe61501-7615-41b8-abcd-d4b7cfe63183 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:15.257673 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe61501-7615-41b8-abcd-d4b7cfe63183 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:15.259128 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe61501-7615-41b8-abcd-d4b7cfe63183 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:15.269281 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:15.274489 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-9fe61501-7615-41b8-abcd-d4b7cfe63183 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:19:15.274489 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:19:15.274489 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:19:15.274489 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:19:15.274489 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:19:15.274489 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:19:15.274489 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:19:15.274489 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:19:15.274489 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:19:15.274489 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:19:15.274489 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:19:15.274489 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:19:15.274489 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:19:15.274489 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:19:15.274489 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:19:15.274489 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:19:15.275431 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:19:15.275431 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:19:15.275431 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:19:15.275431 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:19:15.275431 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:19:15.275431 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:19:15.275431 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:19:15.275431 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:19:15.275431 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:19:15.275431 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:19:15.275431 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:19:15.275431 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:19:15.275431 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:19:15.275431 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:19:15.275431 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:19:15.275431 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:19:15.275431 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:19:15.276346 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:19:15.276346 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:19:15.276346 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:19:15.276346 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 515/1034] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:19:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:19:15.281614 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe61501-7615-41b8-abcd-d4b7cfe63183 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:15.282007 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4025bbe4-84e4-4c51-b1ba-02e526307505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:15.283491 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4025bbe4-84e4-4c51-b1ba-02e526307505 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:15.283984 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4025bbe4-84e4-4c51-b1ba-02e526307505 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:15.284454 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4025bbe4-84e4-4c51-b1ba-02e526307505 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:15.286694 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4025bbe4-84e4-4c51-b1ba-02e526307505 None admin] RBAC: Adding query filter params (interface=public, service_id=15a411533f9a4c07a86ee7871512ec9e, region_id=RegionOne) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:19:15.287294 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4025bbe4-84e4-4c51-b1ba-02e526307505 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:15.288881 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4025bbe4-84e4-4c51-b1ba-02e526307505 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:15.295593 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 520/1035] 10.222.0.102 () {56 vars in 1256 bytes} [Thu Sep 1 17:19:15 2022] GET /identity/v3/endpoints?service_id=15a411533f9a4c07a86ee7871512ec9e&interface=public®ion_id=RegionOne => generated 195 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:19:15.301326 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4025bbe4-84e4-4c51-b1ba-02e526307505 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:15.394129 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe61501-7615-41b8-abcd-d4b7cfe63183 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:15.412280 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4025bbe4-84e4-4c51-b1ba-02e526307505 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:15.430395 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59ca9cc-d4e5-4aba-b018-337647f58fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:15.431026 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59ca9cc-d4e5-4aba-b018-337647f58fd6 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:15.431309 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59ca9cc-d4e5-4aba-b018-337647f58fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:15.431583 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59ca9cc-d4e5-4aba-b018-337647f58fd6 None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:15.432239 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59ca9cc-d4e5-4aba-b018-337647f58fd6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:19:15.432543 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59ca9cc-d4e5-4aba-b018-337647f58fd6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:15.434007 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59ca9cc-d4e5-4aba-b018-337647f58fd6 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:15.447556 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 516/1036] 10.222.0.102 () {56 vars in 1114 bytes} [Thu Sep 1 17:19:15 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:19:15.451090 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ffd449-9b55-46ed-a61f-7fec0a53ab83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:15.451904 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ffd449-9b55-46ed-a61f-7fec0a53ab83 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:15.452390 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ffd449-9b55-46ed-a61f-7fec0a53ab83 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:15.452838 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ffd449-9b55-46ed-a61f-7fec0a53ab83 None admin] PATH_INFO: `/v3/services` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:15.453241 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59ca9cc-d4e5-4aba-b018-337647f58fd6 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:15.453929 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ffd449-9b55-46ed-a61f-7fec0a53ab83 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:15.455592 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ffd449-9b55-46ed-a61f-7fec0a53ab83 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:15.464169 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 521/1037] 10.222.0.102 () {56 vars in 1092 bytes} [Thu Sep 1 17:19:15 2022] GET /identity/v3/services => generated 1966 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 17:19:15.497119 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:15.573429 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59ca9cc-d4e5-4aba-b018-337647f58fd6 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:15.619101 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b64e900-51e4-4d10-b43f-113675c9610d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:15.619738 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b64e900-51e4-4d10-b43f-113675c9610d None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:15.620020 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b64e900-51e4-4d10-b43f-113675c9610d None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:15.620284 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b64e900-51e4-4d10-b43f-113675c9610d None admin] PATH_INFO: `/v3/users/nova` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:15.628528 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b64e900-51e4-4d10-b43f-113675c9610d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:15.630091 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b64e900-51e4-4d10-b43f-113675c9610d None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:15.647833 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-3b64e900-51e4-4d10-b43f-113675c9610d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 17:19:15.647833 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:19:15.647833 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:19:15.647833 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:19:15.647833 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:19:15.647833 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:19:15.647833 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:19:15.647833 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:19:15.647833 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:19:15.647833 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:19:15.647833 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:19:15.647833 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:19:15.647833 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:19:15.647833 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:19:15.647833 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:19:15.647833 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:19:15.648807 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:19:15.648807 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:19:15.648807 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:19:15.648807 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:19:15.648807 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:19:15.648807 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:19:15.648807 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:19:15.648807 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:19:15.648807 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:19:15.648807 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:19:15.648807 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:19:15.648807 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:19:15.648807 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:19:15.648807 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:19:15.648807 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:19:15.648807 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:19:15.648807 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:19:15.648807 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:19:15.650203 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:19:15.650203 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:19:15.650203 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:19:15.650203 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:19:15.650203 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:19:15.650203 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:19:15.650203 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:19:15.650203 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:19:15.650203 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 17:19:15.650203 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:19:15.650203 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 517/1038] 10.222.0.102 () {56 vars in 1098 bytes} [Thu Sep 1 17:19:15 2022] GET /identity/v3/users/nova => generated 82 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:19:15.655047 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ffd449-9b55-46ed-a61f-7fec0a53ab83 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:15.712890 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:15.758704 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ffd449-9b55-46ed-a61f-7fec0a53ab83 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:15.792741 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba491946-55ae-46ad-824f-41b953695465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:15.793369 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba491946-55ae-46ad-824f-41b953695465 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:15.793651 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba491946-55ae-46ad-824f-41b953695465 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:15.793918 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba491946-55ae-46ad-824f-41b953695465 None admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:15.794638 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba491946-55ae-46ad-824f-41b953695465 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:19:15.794939 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba491946-55ae-46ad-824f-41b953695465 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:15.796359 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba491946-55ae-46ad-824f-41b953695465 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:15.815738 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 522/1039] 10.222.0.102 () {56 vars in 1138 bytes} [Thu Sep 1 17:19:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 370 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:19:15.821760 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b64e900-51e4-4d10-b43f-113675c9610d None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:15.928171 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:15.938840 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b64e900-51e4-4d10-b43f-113675c9610d None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:15.975232 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b6c62d-6df4-4f75-af53-4125fda91cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:15.975856 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b6c62d-6df4-4f75-af53-4125fda91cb1 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:15.976138 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b6c62d-6df4-4f75-af53-4125fda91cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:15.976411 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b6c62d-6df4-4f75-af53-4125fda91cb1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:16.044648 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b6c62d-6df4-4f75-af53-4125fda91cb1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:16.046128 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b6c62d-6df4-4f75-af53-4125fda91cb1 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:16.140213 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:16.151039 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 518/1040] 10.222.0.102 () {58 vars in 1287 bytes} [Thu Sep 1 17:19:15 2022] GET /identity/v3/auth/tokens => generated 1973 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 01 17:19:16.156578 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba491946-55ae-46ad-824f-41b953695465 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:16.261478 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba491946-55ae-46ad-824f-41b953695465 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:16.275244 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:16.282295 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398e6fab-c4e8-4adb-9230-e6797b29fafc None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:16.283177 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398e6fab-c4e8-4adb-9230-e6797b29fafc None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:16.283514 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398e6fab-c4e8-4adb-9230-e6797b29fafc None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:16.285074 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 519/1041] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:19:16 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Sep 01 17:19:16.290259 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f118e6-e43d-4de7-bc44-eaa883230946 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:16.290613 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f118e6-e43d-4de7-bc44-eaa883230946 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:16.291045 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f118e6-e43d-4de7-bc44-eaa883230946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:16.298981 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e019eb-c92b-4a09-b771-5814d919d68c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:16.299612 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e019eb-c92b-4a09-b771-5814d919d68c None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:16.299884 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e019eb-c92b-4a09-b771-5814d919d68c None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:16.300145 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e019eb-c92b-4a09-b771-5814d919d68c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:16.309331 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-67f118e6-e43d-4de7-bc44-eaa883230946 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:19:16.310527 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-67f118e6-e43d-4de7-bc44-eaa883230946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:19:16.310635 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e019eb-c92b-4a09-b771-5814d919d68c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:16.312006 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e019eb-c92b-4a09-b771-5814d919d68c None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:16.327133 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-28e019eb-c92b-4a09-b771-5814d919d68c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:19:16.327133 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:19:16.327133 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:19:16.327133 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:19:16.327133 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:19:16.327133 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:19:16.327133 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:19:16.327133 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:19:16.327133 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:19:16.327133 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:19:16.327133 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:19:16.327133 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:19:16.327133 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:19:16.327133 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:19:16.327133 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:19:16.327133 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:19:16.328093 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:19:16.328093 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:19:16.328093 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:19:16.328093 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:19:16.328093 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:19:16.328093 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:19:16.328093 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:19:16.328093 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:19:16.328093 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:19:16.328093 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:19:16.328093 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:19:16.328093 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:19:16.328093 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:19:16.328093 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:19:16.328093 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:19:16.328093 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:19:16.328093 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:19:16.328990 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:19:16.328990 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:19:16.328990 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:19:16.328990 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 523/1042] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:19:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:19:16.333910 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e019eb-c92b-4a09-b771-5814d919d68c None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:16.386562 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 520/1043] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:19:16 2022] POST /identity/v3/auth/tokens => generated 1973 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 17:19:16.449187 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e019eb-c92b-4a09-b771-5814d919d68c None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:16.483926 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcef09ed-551b-4674-a09f-4206034ca06f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:16.484530 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcef09ed-551b-4674-a09f-4206034ca06f None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:16.484807 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcef09ed-551b-4674-a09f-4206034ca06f None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:16.485075 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcef09ed-551b-4674-a09f-4206034ca06f None admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:16.485738 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcef09ed-551b-4674-a09f-4206034ca06f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:19:16.486656 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcef09ed-551b-4674-a09f-4206034ca06f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:16.488006 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcef09ed-551b-4674-a09f-4206034ca06f None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:16.500193 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 524/1044] 10.222.0.102 () {56 vars in 1114 bytes} [Thu Sep 1 17:19:16 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:19:16.506137 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f118e6-e43d-4de7-bc44-eaa883230946 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:16.609420 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f118e6-e43d-4de7-bc44-eaa883230946 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:16.633524 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:16.646192 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298b3a64-a06c-4e55-99bf-8f64d6ce5c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:16.646915 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298b3a64-a06c-4e55-99bf-8f64d6ce5c80 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:16.647193 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298b3a64-a06c-4e55-99bf-8f64d6ce5c80 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:16.647462 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298b3a64-a06c-4e55-99bf-8f64d6ce5c80 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:16.654017 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298b3a64-a06c-4e55-99bf-8f64d6ce5c80 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:16.655465 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298b3a64-a06c-4e55-99bf-8f64d6ce5c80 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:16.660796 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcef09ed-551b-4674-a09f-4206034ca06f None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:16.667322 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-298b3a64-a06c-4e55-99bf-8f64d6ce5c80 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:19:16.667322 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:19:16.667322 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:19:16.667322 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:19:16.667322 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:19:16.667322 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:19:16.667322 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:19:16.667322 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:19:16.667322 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:19:16.667322 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:19:16.667322 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:19:16.667322 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:19:16.667322 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:19:16.667322 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:19:16.667322 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:19:16.668240 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:19:16.668240 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:19:16.668240 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:19:16.668240 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:19:16.668240 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:19:16.668240 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:19:16.668240 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:19:16.668240 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:19:16.668240 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:19:16.668240 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:19:16.668240 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:19:16.668240 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:19:16.668240 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:19:16.668240 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:19:16.668240 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:19:16.668240 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:19:16.668240 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:19:16.669133 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:19:16.669133 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:19:16.669133 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:19:16.669133 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:19:16.669133 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:19:16.669133 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:19:16.669133 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:19:16.669133 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:19:16.673826 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 521/1045] 10.222.0.102 () {56 vars in 1116 bytes} [Thu Sep 1 17:19:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:19:16.674491 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298b3a64-a06c-4e55-99bf-8f64d6ce5c80 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:16.687918 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:16.766337 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcef09ed-551b-4674-a09f-4206034ca06f None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:16.776379 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298b3a64-a06c-4e55-99bf-8f64d6ce5c80 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:16.800804 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aac25e8-d385-4986-8935-9d3b9e4a466f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:16.801457 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aac25e8-d385-4986-8935-9d3b9e4a466f None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:16.801740 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aac25e8-d385-4986-8935-9d3b9e4a466f None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:16.802017 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aac25e8-d385-4986-8935-9d3b9e4a466f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:16.815118 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcf15b5-cc17-4c2d-ace2-0add1402d952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:16.815274 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aac25e8-d385-4986-8935-9d3b9e4a466f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:19:16.815733 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcf15b5-cc17-4c2d-ace2-0add1402d952 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:16.816010 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcf15b5-cc17-4c2d-ace2-0add1402d952 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:16.816280 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcf15b5-cc17-4c2d-ace2-0add1402d952 None admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:16.816979 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcf15b5-cc17-4c2d-ace2-0add1402d952 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:19:16.817284 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcf15b5-cc17-4c2d-ace2-0add1402d952 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:16.818837 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcf15b5-cc17-4c2d-ace2-0add1402d952 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:16.820047 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-6aac25e8-d385-4986-8935-9d3b9e4a466f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:19:16.820047 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:19:16.820047 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:19:16.820047 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:19:16.820047 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:19:16.820047 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:19:16.820047 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:19:16.820047 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:19:16.820047 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:19:16.820047 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:19:16.820047 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:19:16.820047 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:19:16.820047 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:19:16.820047 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:19:16.820047 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:19:16.820047 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:19:16.820953 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:19:16.820953 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:19:16.820953 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:19:16.820953 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:19:16.820953 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:19:16.820953 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:19:16.820953 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:19:16.820953 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:19:16.820953 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:19:16.820953 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:19:16.820953 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:19:16.820953 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:19:16.820953 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:19:16.820953 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:19:16.820953 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:19:16.820953 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:19:16.822310 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:19:16.822310 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:19:16.822310 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:19:16.822310 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:19:16.822310 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:19:16.822310 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:19:16.822310 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:19:16.822310 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:19:16.822310 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:19:16.822310 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:19:16.822310 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:19:16.822310 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:19:16.822310 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 525/1046] 10.222.0.102 () {56 vars in 1101 bytes} [Thu Sep 1 17:19:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:19:16.827056 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aac25e8-d385-4986-8935-9d3b9e4a466f None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:16.834207 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 522/1047] 10.222.0.102 () {56 vars in 1153 bytes} [Thu Sep 1 17:19:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:19:16.839917 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcf15b5-cc17-4c2d-ace2-0add1402d952 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:16.892915 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:16.921261 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:16.945135 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aac25e8-d385-4986-8935-9d3b9e4a466f None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:16.956838 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcf15b5-cc17-4c2d-ace2-0add1402d952 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:16.982714 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6945abd-c8a9-4b78-ba1e-31cad19d5cdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:16.983324 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6945abd-c8a9-4b78-ba1e-31cad19d5cdf None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:16.983606 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6945abd-c8a9-4b78-ba1e-31cad19d5cdf None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:16.983879 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6945abd-c8a9-4b78-ba1e-31cad19d5cdf None admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:16.984539 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6945abd-c8a9-4b78-ba1e-31cad19d5cdf None admin] RBAC: Adding query filter params (name=admin) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:19:16.984844 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6945abd-c8a9-4b78-ba1e-31cad19d5cdf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:16.986284 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6945abd-c8a9-4b78-ba1e-31cad19d5cdf None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:16.993923 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bd8c22-5008-4611-9dd4-854e2d0598a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:16.994737 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 526/1048] 10.222.0.102 () {56 vars in 1104 bytes} [Thu Sep 1 17:19:16 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:19:16.994981 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bd8c22-5008-4611-9dd4-854e2d0598a0 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:16.995437 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bd8c22-5008-4611-9dd4-854e2d0598a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:16.995993 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bd8c22-5008-4611-9dd4-854e2d0598a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:16.996892 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bd8c22-5008-4611-9dd4-854e2d0598a0 None admin] RBAC: Adding query filter params (role.id=9329b00eca174ca5a29094e84e243fa4, scope.project.id=e78fff7f1f6e4f68b8d233ad2e5bbd4f, user.id=5f8293316baf4774b62fc11d55d50865) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:19:16.997419 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bd8c22-5008-4611-9dd4-854e2d0598a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:16.999021 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bd8c22-5008-4611-9dd4-854e2d0598a0 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:17.000598 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6945abd-c8a9-4b78-ba1e-31cad19d5cdf None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:17.005775 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 523/1049] 10.222.0.102 () {56 vars in 1379 bytes} [Thu Sep 1 17:19:16 2022] GET /identity/v3/role_assignments?user.id=5f8293316baf4774b62fc11d55d50865&scope.project.id=e78fff7f1f6e4f68b8d233ad2e5bbd4f&role.id=9329b00eca174ca5a29094e84e243fa4 => generated 260 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:19:17.124811 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6945abd-c8a9-4b78-ba1e-31cad19d5cdf None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:17.165834 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24be0523-652f-4acf-9d6d-b43e18f6e4de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:17.166391 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24be0523-652f-4acf-9d6d-b43e18f6e4de None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:17.166670 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24be0523-652f-4acf-9d6d-b43e18f6e4de None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:17.166945 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24be0523-652f-4acf-9d6d-b43e18f6e4de None admin] PATH_INFO: `/v3/users/3f85c02d50424a14a44a2eaa5e3c5fff` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:17.181851 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24be0523-652f-4acf-9d6d-b43e18f6e4de None admin] RBAC: Authorizing `identity:get_user(user_id=3f85c02d50424a14a44a2eaa5e3c5fff)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:17.183298 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24be0523-652f-4acf-9d6d-b43e18f6e4de None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:17.184970 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 527/1050] 10.222.0.102 () {56 vars in 1182 bytes} [Thu Sep 1 17:19:16 2022] GET /identity/v3/users/3f85c02d50424a14a44a2eaa5e3c5fff => generated 286 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:19:17.191227 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bd8c22-5008-4611-9dd4-854e2d0598a0 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:17.295262 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bd8c22-5008-4611-9dd4-854e2d0598a0 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:17.302610 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:17.331421 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926baf49-3350-4919-9ff3-fefcd16e04fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:17.332036 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926baf49-3350-4919-9ff3-fefcd16e04fa None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:17.332314 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926baf49-3350-4919-9ff3-fefcd16e04fa None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:17.332586 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926baf49-3350-4919-9ff3-fefcd16e04fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:17.401678 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926baf49-3350-4919-9ff3-fefcd16e04fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:17.403139 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926baf49-3350-4919-9ff3-fefcd16e04fa None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:17.445675 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab87679-4377-45c3-8802-f8546aa0fbf5 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:17.445675 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab87679-4377-45c3-8802-f8546aa0fbf5 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:17.445675 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab87679-4377-45c3-8802-f8546aa0fbf5 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:17.446089 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 528/1051] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:19:17 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Sep 01 17:19:17.451184 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb50a416-854a-4e95-a392-ac1087048b75 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:17.451471 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb50a416-854a-4e95-a392-ac1087048b75 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:17.451751 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb50a416-854a-4e95-a392-ac1087048b75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:17.459552 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:17.472392 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-bb50a416-854a-4e95-a392-ac1087048b75 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:19:17.473624 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-bb50a416-854a-4e95-a392-ac1087048b75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:19:17.520860 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 524/1052] 10.222.0.102 () {58 vars in 1287 bytes} [Thu Sep 1 17:19:17 2022] GET /identity/v3/auth/tokens => generated 1973 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 17:19:17.522016 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926baf49-3350-4919-9ff3-fefcd16e04fa None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:17.530465 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:17.547153 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 529/1053] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:19:17 2022] POST /identity/v3/auth/tokens => generated 1973 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 17:19:17.624617 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926baf49-3350-4919-9ff3-fefcd16e04fa None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:17.661865 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b29649-8b30-4578-b471-1900bbb106ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:17.662369 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b29649-8b30-4578-b471-1900bbb106ee None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:17.662631 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b29649-8b30-4578-b471-1900bbb106ee None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:17.662902 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b29649-8b30-4578-b471-1900bbb106ee None admin] PATH_INFO: `/v3/projects/8c162948f1a446698552d4407da48267` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:17.663999 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b29649-8b30-4578-b471-1900bbb106ee None admin] RBAC: Authorizing `identity:get_project(project_id=8c162948f1a446698552d4407da48267)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:17.665442 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b29649-8b30-4578-b471-1900bbb106ee None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:17.671954 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 525/1054] 10.222.0.102 () {56 vars in 1191 bytes} [Thu Sep 1 17:19:17 2022] GET /identity/v3/projects/8c162948f1a446698552d4407da48267 => generated 312 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:19:17.673107 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb50a416-854a-4e95-a392-ac1087048b75 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:17.774959 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb50a416-854a-4e95-a392-ac1087048b75 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:17.814325 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45425f22-850a-434a-99f6-e1d7db828b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:17.814944 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45425f22-850a-434a-99f6-e1d7db828b43 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:17.815226 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45425f22-850a-434a-99f6-e1d7db828b43 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:17.815501 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45425f22-850a-434a-99f6-e1d7db828b43 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:17.816226 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45425f22-850a-434a-99f6-e1d7db828b43 None admin] RBAC: Adding query filter params (role.id=36577a6f46d642d4b81805ede9872490, scope.project.id=8c162948f1a446698552d4407da48267, user.id=3f85c02d50424a14a44a2eaa5e3c5fff) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:19:17.816537 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45425f22-850a-434a-99f6-e1d7db828b43 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:17.818454 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45425f22-850a-434a-99f6-e1d7db828b43 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:17.819423 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b29649-8b30-4578-b471-1900bbb106ee None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:17.820212 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:17.825555 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 530/1055] 10.222.0.102 () {56 vars in 1379 bytes} [Thu Sep 1 17:19:17 2022] GET /identity/v3/role_assignments?user.id=3f85c02d50424a14a44a2eaa5e3c5fff&scope.project.id=8c162948f1a446698552d4407da48267&role.id=36577a6f46d642d4b81805ede9872490 => generated 260 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:19:17.922133 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:17.951997 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b29649-8b30-4578-b471-1900bbb106ee None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:17.992550 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634e4451-276b-438d-8e03-81fd1ac940aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:17.993184 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634e4451-276b-438d-8e03-81fd1ac940aa None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:17.993505 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634e4451-276b-438d-8e03-81fd1ac940aa None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:17.993778 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634e4451-276b-438d-8e03-81fd1ac940aa None admin] PATH_INFO: `/v3/services/image` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:17.994363 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634e4451-276b-438d-8e03-81fd1ac940aa None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:17.995729 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634e4451-276b-438d-8e03-81fd1ac940aa None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:18.005180 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-634e4451-276b-438d-8e03-81fd1ac940aa None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 17:19:18.005180 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:19:18.005180 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 17:19:18.005180 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 17:19:18.005180 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 17:19:18.005180 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 17:19:18.005180 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 17:19:18.005180 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:19:18.005180 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 17:19:18.005180 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:19:18.005180 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 17:19:18.005180 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:19:18.005180 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:19:18.005180 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:19:18.005180 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:19:18.005180 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:19:18.005180 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:19:18.006231 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:19:18.006231 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:19:18.006231 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:19:18.006231 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:19:18.006231 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:19:18.006231 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:19:18.006231 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 17:19:18.006231 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 17:19:18.006231 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 17:19:18.006231 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 17:19:18.006231 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:19:18.006231 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:19:18.006231 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:19:18.006231 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:19:18.006231 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:19:18.006231 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:19:18.007104 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:19:18.007104 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:19:18.007104 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:19:18.007104 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:19:18.007104 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:19:18.007104 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:19:18.007104 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:19:18.007104 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:19:18.007104 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:19:18.007104 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:19:18.007104 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 17:19:18.007104 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:19:18.007104 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 17:19:18.007104 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:19:18.007104 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 526/1056] 10.222.0.102 () {56 vars in 1110 bytes} [Thu Sep 1 17:19:17 2022] GET /identity/v3/services/image => generated 86 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:19:18.013337 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45425f22-850a-434a-99f6-e1d7db828b43 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:18.120908 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45425f22-850a-434a-99f6-e1d7db828b43 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:18.154772 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70c9fd6-761f-4075-9ae2-d31056b96b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:18.155380 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70c9fd6-761f-4075-9ae2-d31056b96b1f None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:18.155661 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70c9fd6-761f-4075-9ae2-d31056b96b1f None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:18.155932 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70c9fd6-761f-4075-9ae2-d31056b96b1f None admin] PATH_INFO: `/v3/services` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:18.156555 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70c9fd6-761f-4075-9ae2-d31056b96b1f None admin] RBAC: Adding query filter params (name=image) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:19:18.156872 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70c9fd6-761f-4075-9ae2-d31056b96b1f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:18.158288 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70c9fd6-761f-4075-9ae2-d31056b96b1f None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:18.163497 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 531/1057] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:19:18 2022] GET /identity/v3/services?name=image => generated 123 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:19:18.169099 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634e4451-276b-438d-8e03-81fd1ac940aa None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:18.275348 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634e4451-276b-438d-8e03-81fd1ac940aa None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:18.309081 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282708f7-22bf-4ecb-addf-74412dd5106b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:18.309727 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282708f7-22bf-4ecb-addf-74412dd5106b None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:18.310008 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282708f7-22bf-4ecb-addf-74412dd5106b None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:18.310280 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282708f7-22bf-4ecb-addf-74412dd5106b None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:18.310904 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282708f7-22bf-4ecb-addf-74412dd5106b None admin] RBAC: Adding query filter params (type=image) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:19:18.311207 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282708f7-22bf-4ecb-addf-74412dd5106b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:18.312569 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282708f7-22bf-4ecb-addf-74412dd5106b None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:18.318137 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 527/1058] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:19:18 2022] GET /identity/v3/services?type=image => generated 352 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:19:18.323754 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70c9fd6-761f-4075-9ae2-d31056b96b1f None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:18.387169 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:18.427931 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70c9fd6-761f-4075-9ae2-d31056b96b1f None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:18.461188 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ddef2f-f658-4519-bac6-458a96bb73eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:18.461817 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ddef2f-f658-4519-bac6-458a96bb73eb None admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:18.462098 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ddef2f-f658-4519-bac6-458a96bb73eb None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:18.462366 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ddef2f-f658-4519-bac6-458a96bb73eb None admin] PATH_INFO: `/v3/endpoints` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:18.463036 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ddef2f-f658-4519-bac6-458a96bb73eb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:18.464394 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ddef2f-f658-4519-bac6-458a96bb73eb None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:18.493341 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 532/1059] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:19:18 2022] POST /identity/v3/endpoints => generated 329 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:19:18.937820 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2270ebf-e0c6-4698-934d-a47817a0bb40 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:18.938341 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2270ebf-e0c6-4698-934d-a47817a0bb40 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:18.938418 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2270ebf-e0c6-4698-934d-a47817a0bb40 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:18.939830 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 528/1060] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:19:18 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:19:18.945126 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd8c525-b6a7-4a78-a3af-89e1e59a3622 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:18.945597 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd8c525-b6a7-4a78-a3af-89e1e59a3622 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:18.945791 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd8c525-b6a7-4a78-a3af-89e1e59a3622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:18.964969 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-cfd8c525-b6a7-4a78-a3af-89e1e59a3622 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:19:18.966197 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-cfd8c525-b6a7-4a78-a3af-89e1e59a3622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:19:19.046064 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 533/1061] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:19:18 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:19:19.319446 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2270ebf-e0c6-4698-934d-a47817a0bb40 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:19.322499 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:19.432147 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2270ebf-e0c6-4698-934d-a47817a0bb40 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:19.469037 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be24ce8-5776-49f6-bea7-51a43d5c5ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:19.469713 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be24ce8-5776-49f6-bea7-51a43d5c5ba2 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:19.470016 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be24ce8-5776-49f6-bea7-51a43d5c5ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:19.470762 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be24ce8-5776-49f6-bea7-51a43d5c5ba2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:19.482343 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be24ce8-5776-49f6-bea7-51a43d5c5ba2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:19:19.487710 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-6be24ce8-5776-49f6-bea7-51a43d5c5ba2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 17:19:19.487710 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:19:19.487710 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:19:19.487710 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:19:19.487710 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:19:19.487710 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:19:19.487710 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:19:19.487710 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:19:19.487710 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:19:19.487710 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:19:19.487710 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:19:19.487710 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:19:19.487710 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:19:19.487710 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:19:19.487710 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:19:19.487710 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:19:19.488610 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:19:19.488610 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:19:19.488610 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:19:19.488610 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:19:19.488610 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:19:19.488610 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:19:19.488610 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:19:19.488610 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:19:19.488610 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:19:19.488610 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:19:19.488610 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:19:19.488610 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:19:19.488610 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:19:19.488610 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:19:19.488610 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:19:19.488610 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:19:19.490011 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:19:19.490011 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:19:19.490011 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:19:19.490011 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:19:19.490011 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:19:19.490011 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:19:19.490011 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:19:19.490011 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:19:19.490011 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:19:19.490011 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:19:19.490011 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 17:19:19.490011 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:19:19.490934 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 529/1062] 10.222.0.102 () {56 vars in 1125 bytes} [Thu Sep 1 17:19:19 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:19:19.495050 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd8c525-b6a7-4a78-a3af-89e1e59a3622 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:19.543301 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:19.601791 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd8c525-b6a7-4a78-a3af-89e1e59a3622 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:19.636389 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e3cab0-c45d-44d0-8dae-7dc498b58428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:19.636995 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e3cab0-c45d-44d0-8dae-7dc498b58428 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:19.637271 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e3cab0-c45d-44d0-8dae-7dc498b58428 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:19.637566 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e3cab0-c45d-44d0-8dae-7dc498b58428 None admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:19.638224 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e3cab0-c45d-44d0-8dae-7dc498b58428 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:19:19.638524 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e3cab0-c45d-44d0-8dae-7dc498b58428 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:19.639916 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e3cab0-c45d-44d0-8dae-7dc498b58428 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:19.648786 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 534/1063] 10.222.0.102 () {56 vars in 1120 bytes} [Thu Sep 1 17:19:19 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 340 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:19:19.654257 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be24ce8-5776-49f6-bea7-51a43d5c5ba2 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:19.716182 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:19.756805 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be24ce8-5776-49f6-bea7-51a43d5c5ba2 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:19.784339 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cbf71a-93f7-49d3-bf7e-5a635ecda3a2 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:19.784630 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cbf71a-93f7-49d3-bf7e-5a635ecda3a2 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:19.784909 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cbf71a-93f7-49d3-bf7e-5a635ecda3a2 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:19.786293 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 535/1064] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:19:19 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Sep 01 17:19:19.791128 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751646f5-2701-4712-86c6-f58bc902eda0 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:19.791419 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751646f5-2701-4712-86c6-f58bc902eda0 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:19.791693 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751646f5-2701-4712-86c6-f58bc902eda0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:19.792014 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2f0169-962f-4382-ab3b-1058220d74d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:19.792639 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2f0169-962f-4382-ab3b-1058220d74d8 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:19.792917 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2f0169-962f-4382-ab3b-1058220d74d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:19.793194 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2f0169-962f-4382-ab3b-1058220d74d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:19.803991 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2f0169-962f-4382-ab3b-1058220d74d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:19.805467 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2f0169-962f-4382-ab3b-1058220d74d8 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:19.810152 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-751646f5-2701-4712-86c6-f58bc902eda0 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:19:19.812902 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-751646f5-2701-4712-86c6-f58bc902eda0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:19:19.823293 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-fa2f0169-962f-4382-ab3b-1058220d74d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:19:19.823293 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:19:19.823293 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:19:19.823293 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:19:19.823293 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:19:19.823293 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:19:19.823293 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:19:19.823293 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:19:19.823293 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:19:19.823293 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:19:19.823293 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:19:19.823293 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:19:19.823293 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:19:19.823293 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:19:19.823293 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:19:19.823293 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:19:19.824312 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:19:19.824312 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:19:19.824312 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:19:19.824312 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:19:19.824312 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:19:19.824312 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:19:19.824312 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:19:19.824312 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:19:19.824312 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:19:19.824312 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:19:19.824312 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:19:19.824312 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:19:19.824312 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:19:19.824312 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:19:19.824312 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:19:19.824312 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:19:19.824312 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:19:19.825206 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:19:19.825206 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:19:19.825206 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:19:19.825206 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 530/1065] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:19:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:19:19.830084 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2f0169-962f-4382-ab3b-1058220d74d8 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:19.897458 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 536/1066] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:19:19 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 17:19:19.943336 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2f0169-962f-4382-ab3b-1058220d74d8 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:19.957163 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:19.979947 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f1577c-6d0c-412d-85c7-a8a2aff71c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:19.980552 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f1577c-6d0c-412d-85c7-a8a2aff71c94 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:19.980828 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f1577c-6d0c-412d-85c7-a8a2aff71c94 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:19.981095 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f1577c-6d0c-412d-85c7-a8a2aff71c94 None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:19.981755 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f1577c-6d0c-412d-85c7-a8a2aff71c94 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:19:19.982056 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f1577c-6d0c-412d-85c7-a8a2aff71c94 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:19.983412 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f1577c-6d0c-412d-85c7-a8a2aff71c94 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:19.997788 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 531/1067] 10.222.0.102 () {56 vars in 1114 bytes} [Thu Sep 1 17:19:19 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:19:20.005407 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751646f5-2701-4712-86c6-f58bc902eda0 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:20.041017 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:20.108717 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751646f5-2701-4712-86c6-f58bc902eda0 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:20.143044 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cbb9b7-7fa2-47f0-aa5c-572cd4f1949f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:20.143668 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cbb9b7-7fa2-47f0-aa5c-572cd4f1949f None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:20.143944 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cbb9b7-7fa2-47f0-aa5c-572cd4f1949f None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:20.144209 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cbb9b7-7fa2-47f0-aa5c-572cd4f1949f None admin] PATH_INFO: `/v3/users/nova` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:20.151332 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cbb9b7-7fa2-47f0-aa5c-572cd4f1949f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:20.152692 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cbb9b7-7fa2-47f0-aa5c-572cd4f1949f None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:20.164938 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-12cbb9b7-7fa2-47f0-aa5c-572cd4f1949f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 17:19:20.164938 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:19:20.164938 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:19:20.164938 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:19:20.164938 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:19:20.164938 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:19:20.164938 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:19:20.164938 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:19:20.164938 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:19:20.164938 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:19:20.164938 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:19:20.164938 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:19:20.164938 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:19:20.164938 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:19:20.164938 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:19:20.164938 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:19:20.166421 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:19:20.166421 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:19:20.166421 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:19:20.166421 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:19:20.166421 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:19:20.166421 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:19:20.166421 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:19:20.166421 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:19:20.166421 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:19:20.166421 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:19:20.166421 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:19:20.166421 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:19:20.166421 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:19:20.166421 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:19:20.166421 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:19:20.166421 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:19:20.166421 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:19:20.166421 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:19:20.167770 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:19:20.167770 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:19:20.167770 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:19:20.167770 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:19:20.167770 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:19:20.167770 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:19:20.167770 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:19:20.167770 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:19:20.167770 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 17:19:20.167770 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:19:20.168336 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f1577c-6d0c-412d-85c7-a8a2aff71c94 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:20.169704 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 537/1068] 10.222.0.102 () {56 vars in 1098 bytes} [Thu Sep 1 17:19:20 2022] GET /identity/v3/users/nova => generated 82 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:19:20.172245 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cbb9b7-7fa2-47f0-aa5c-572cd4f1949f None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:20.281764 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:20.283352 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f1577c-6d0c-412d-85c7-a8a2aff71c94 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:20.289703 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cbb9b7-7fa2-47f0-aa5c-572cd4f1949f None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:20.320248 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da93f59-c01a-4a22-8b70-8fb19edf4a3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:20.321173 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da93f59-c01a-4a22-8b70-8fb19edf4a3a None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:20.321285 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da93f59-c01a-4a22-8b70-8fb19edf4a3a None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:20.321522 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da93f59-c01a-4a22-8b70-8fb19edf4a3a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:20.327394 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7261e20c-4679-4baa-aa9a-4161f6811e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:20.328004 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7261e20c-4679-4baa-aa9a-4161f6811e62 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:20.328287 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7261e20c-4679-4baa-aa9a-4161f6811e62 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:20.328554 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7261e20c-4679-4baa-aa9a-4161f6811e62 None admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:20.329282 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7261e20c-4679-4baa-aa9a-4161f6811e62 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:19:20.330393 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7261e20c-4679-4baa-aa9a-4161f6811e62 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:20.331625 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7261e20c-4679-4baa-aa9a-4161f6811e62 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:20.333594 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da93f59-c01a-4a22-8b70-8fb19edf4a3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:19:20.338449 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-4da93f59-c01a-4a22-8b70-8fb19edf4a3a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:19:20.338449 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:19:20.338449 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:19:20.338449 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:19:20.338449 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:19:20.338449 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:19:20.338449 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:19:20.338449 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:19:20.338449 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:19:20.338449 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:19:20.338449 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:19:20.338449 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:19:20.338449 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:19:20.338449 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:19:20.338449 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:19:20.338449 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:19:20.339361 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:19:20.339361 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:19:20.339361 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:19:20.339361 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:19:20.339361 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:19:20.339361 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:19:20.339361 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:19:20.339361 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:19:20.339361 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:19:20.339361 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:19:20.339361 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:19:20.339361 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:19:20.339361 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:19:20.339361 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:19:20.339361 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:19:20.339361 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:19:20.340292 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:19:20.340292 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:19:20.340292 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:19:20.340292 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:19:20.340292 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:19:20.340292 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:19:20.340292 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:19:20.340292 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:19:20.340292 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:19:20.340292 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:19:20.340292 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:19:20.340292 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:19:20.340292 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 532/1069] 10.222.0.102 () {56 vars in 1101 bytes} [Thu Sep 1 17:19:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:19:20.346205 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da93f59-c01a-4a22-8b70-8fb19edf4a3a None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:20.358037 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 538/1070] 10.222.0.102 () {56 vars in 1138 bytes} [Thu Sep 1 17:19:20 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 370 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:19:20.359084 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7261e20c-4679-4baa-aa9a-4161f6811e62 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:20.385280 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:20.461397 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da93f59-c01a-4a22-8b70-8fb19edf4a3a None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:20.464349 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7261e20c-4679-4baa-aa9a-4161f6811e62 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:20.496710 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31090dd-10b6-4692-8b9d-536ed224df48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:20.497347 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31090dd-10b6-4692-8b9d-536ed224df48 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:20.497642 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31090dd-10b6-4692-8b9d-536ed224df48 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:20.497913 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31090dd-10b6-4692-8b9d-536ed224df48 None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:20.498585 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31090dd-10b6-4692-8b9d-536ed224df48 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:19:20.498761 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21af8ee7-dead-4c2d-aa69-d855a5da1338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:20.498874 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31090dd-10b6-4692-8b9d-536ed224df48 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:20.499302 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21af8ee7-dead-4c2d-aa69-d855a5da1338 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:20.499580 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21af8ee7-dead-4c2d-aa69-d855a5da1338 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:20.499851 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21af8ee7-dead-4c2d-aa69-d855a5da1338 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:20.500248 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31090dd-10b6-4692-8b9d-536ed224df48 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:20.509619 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 533/1071] 10.222.0.102 () {56 vars in 1104 bytes} [Thu Sep 1 17:19:20 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:19:20.510620 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21af8ee7-dead-4c2d-aa69-d855a5da1338 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:20.510985 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f218491-050a-4c9e-a24b-37bbd7df8d4d None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:20.511099 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f218491-050a-4c9e-a24b-37bbd7df8d4d None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:20.511346 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f218491-050a-4c9e-a24b-37bbd7df8d4d None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:20.512430 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21af8ee7-dead-4c2d-aa69-d855a5da1338 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:20.513349 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 534/1072] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:19:20 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:19:20.515156 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f218491-050a-4c9e-a24b-37bbd7df8d4d None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:20.530487 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-21af8ee7-dead-4c2d-aa69-d855a5da1338 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:19:20.530487 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:19:20.530487 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:19:20.530487 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:19:20.530487 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:19:20.530487 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:19:20.530487 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:19:20.530487 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:19:20.530487 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:19:20.530487 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:19:20.530487 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:19:20.530487 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:19:20.530487 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:19:20.530487 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:19:20.530487 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:19:20.530487 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:19:20.531654 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:19:20.531654 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:19:20.531654 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:19:20.531654 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:19:20.531654 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:19:20.531654 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:19:20.531654 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:19:20.531654 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:19:20.531654 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:19:20.531654 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:19:20.531654 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:19:20.531654 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:19:20.531654 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:19:20.531654 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:19:20.531654 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:19:20.531654 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:19:20.531654 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:19:20.532602 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:19:20.532602 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:19:20.532602 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:19:20.532602 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 539/1073] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:19:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:19:20.532834 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac76119-8026-478b-aca3-d040878774fa None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:20.533052 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac76119-8026-478b-aca3-d040878774fa None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:20.533340 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac76119-8026-478b-aca3-d040878774fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:20.546362 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:20.553088 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-3ac76119-8026-478b-aca3-d040878774fa None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:19:20.554329 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-3ac76119-8026-478b-aca3-d040878774fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:19:20.619761 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:20.626094 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 540/1074] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:19:20 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:19:20.627513 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac76119-8026-478b-aca3-d040878774fa None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:20.628226 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f218491-050a-4c9e-a24b-37bbd7df8d4d None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:20.665544 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae668f5-a523-4e6b-91ba-34aa2c97fa3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:20.666183 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae668f5-a523-4e6b-91ba-34aa2c97fa3f None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:20.666456 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae668f5-a523-4e6b-91ba-34aa2c97fa3f None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:20.666716 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae668f5-a523-4e6b-91ba-34aa2c97fa3f None admin] PATH_INFO: `/v3/users/3f85c02d50424a14a44a2eaa5e3c5fff` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:20.667842 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae668f5-a523-4e6b-91ba-34aa2c97fa3f None admin] RBAC: Authorizing `identity:get_user(user_id=3f85c02d50424a14a44a2eaa5e3c5fff)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:20.669337 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae668f5-a523-4e6b-91ba-34aa2c97fa3f None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:20.670924 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 535/1075] 10.222.0.102 () {56 vars in 1182 bytes} [Thu Sep 1 17:19:20 2022] GET /identity/v3/users/3f85c02d50424a14a44a2eaa5e3c5fff => generated 286 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:19:20.676516 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae668f5-a523-4e6b-91ba-34aa2c97fa3f None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:20.742664 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac76119-8026-478b-aca3-d040878774fa None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:20.781556 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2323477b-d0e3-4a82-89c3-ddc4e959e903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:20.782357 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae668f5-a523-4e6b-91ba-34aa2c97fa3f None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:20.782888 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2323477b-d0e3-4a82-89c3-ddc4e959e903 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:20.783319 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2323477b-d0e3-4a82-89c3-ddc4e959e903 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:20.784256 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2323477b-d0e3-4a82-89c3-ddc4e959e903 None admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:20.784911 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2323477b-d0e3-4a82-89c3-ddc4e959e903 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:19:20.785220 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2323477b-d0e3-4a82-89c3-ddc4e959e903 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:20.786612 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2323477b-d0e3-4a82-89c3-ddc4e959e903 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:20.788984 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:20.799390 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 541/1076] 10.222.0.102 () {56 vars in 1114 bytes} [Thu Sep 1 17:19:20 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:19:20.804985 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2323477b-d0e3-4a82-89c3-ddc4e959e903 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:20.822010 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741e0052-e3a6-4f1b-87b3-afeebe26efd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:20.822655 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741e0052-e3a6-4f1b-87b3-afeebe26efd9 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:20.822938 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741e0052-e3a6-4f1b-87b3-afeebe26efd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:20.823218 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741e0052-e3a6-4f1b-87b3-afeebe26efd9 None admin] PATH_INFO: `/v3/projects/8c162948f1a446698552d4407da48267` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:20.824369 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741e0052-e3a6-4f1b-87b3-afeebe26efd9 None admin] RBAC: Authorizing `identity:get_project(project_id=8c162948f1a446698552d4407da48267)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:20.825811 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741e0052-e3a6-4f1b-87b3-afeebe26efd9 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:20.827799 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 536/1077] 10.222.0.102 () {56 vars in 1191 bytes} [Thu Sep 1 17:19:20 2022] GET /identity/v3/projects/8c162948f1a446698552d4407da48267 => generated 312 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:19:20.834195 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741e0052-e3a6-4f1b-87b3-afeebe26efd9 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:20.868222 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:20.926485 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2323477b-d0e3-4a82-89c3-ddc4e959e903 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:20.948900 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741e0052-e3a6-4f1b-87b3-afeebe26efd9 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:20.960256 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d7e4e0-6975-4392-9cf9-7481c58aa3bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:20.960888 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d7e4e0-6975-4392-9cf9-7481c58aa3bf None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:20.961171 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d7e4e0-6975-4392-9cf9-7481c58aa3bf None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:20.961471 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d7e4e0-6975-4392-9cf9-7481c58aa3bf None admin] PATH_INFO: `/v3/projects/service` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:20.968168 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d7e4e0-6975-4392-9cf9-7481c58aa3bf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:20.969618 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d7e4e0-6975-4392-9cf9-7481c58aa3bf None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:20.981607 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-41d7e4e0-6975-4392-9cf9-7481c58aa3bf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:19:20.981607 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:19:20.981607 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:19:20.981607 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:19:20.981607 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:19:20.981607 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:19:20.981607 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:19:20.981607 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:19:20.981607 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:19:20.981607 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:19:20.981607 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:19:20.981607 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:19:20.981607 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:19:20.981607 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:19:20.981607 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:19:20.982993 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:19:20.982993 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:19:20.982993 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:19:20.982993 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:19:20.982993 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:19:20.982993 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:19:20.982993 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:19:20.982993 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:19:20.982993 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:19:20.982993 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:19:20.982993 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:19:20.982993 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:19:20.982993 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:19:20.982993 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:19:20.982993 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:19:20.982993 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:19:20.982993 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:19:20.984698 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:19:20.984698 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:19:20.984698 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:19:20.984698 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:19:20.984698 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:19:20.984698 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:19:20.984698 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:19:20.984698 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:19:20.984698 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 542/1078] 10.222.0.102 () {56 vars in 1116 bytes} [Thu Sep 1 17:19:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:19:20.984698 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d7e4e0-6975-4392-9cf9-7481c58aa3bf None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:20.985241 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5b20d0-0d8d-4b66-919a-1c9492d42089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:20.985660 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5b20d0-0d8d-4b66-919a-1c9492d42089 None admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:20.985813 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5b20d0-0d8d-4b66-919a-1c9492d42089 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:20.986076 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5b20d0-0d8d-4b66-919a-1c9492d42089 None admin] PATH_INFO: `/v3/projects/8c162948f1a446698552d4407da48267/users/3f85c02d50424a14a44a2eaa5e3c5fff/roles/36577a6f46d642d4b81805ede9872490` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:20.988538 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5b20d0-0d8d-4b66-919a-1c9492d42089 None admin] RBAC: Authorizing `identity:create_grant(project_id=8c162948f1a446698552d4407da48267, user_id=3f85c02d50424a14a44a2eaa5e3c5fff, role_id=36577a6f46d642d4b81805ede9872490)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:20.990029 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5b20d0-0d8d-4b66-919a-1c9492d42089 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:21.003290 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 537/1079] 10.222.0.102 () {58 vars in 1444 bytes} [Thu Sep 1 17:19:20 2022] PUT /identity/v3/projects/8c162948f1a446698552d4407da48267/users/3f85c02d50424a14a44a2eaa5e3c5fff/roles/36577a6f46d642d4b81805ede9872490 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:19:21.004039 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5b20d0-0d8d-4b66-919a-1c9492d42089 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:21.047196 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:21.098356 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d7e4e0-6975-4392-9cf9-7481c58aa3bf None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:21.124451 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:21.136058 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5b20d0-0d8d-4b66-919a-1c9492d42089 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:21.138520 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86985c10-6f2d-4bd0-b1c5-071ffc332d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:21.139223 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86985c10-6f2d-4bd0-b1c5-071ffc332d89 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:21.139506 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86985c10-6f2d-4bd0-b1c5-071ffc332d89 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:21.139777 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86985c10-6f2d-4bd0-b1c5-071ffc332d89 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:21.179655 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5f86a8-54b4-4da3-b161-fc4fbebc59af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:21.180555 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5f86a8-54b4-4da3-b161-fc4fbebc59af None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:21.180951 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5f86a8-54b4-4da3-b161-fc4fbebc59af None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:21.181424 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5f86a8-54b4-4da3-b161-fc4fbebc59af None admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:21.182618 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5f86a8-54b4-4da3-b161-fc4fbebc59af None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:19:21.183151 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5f86a8-54b4-4da3-b161-fc4fbebc59af None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:21.185739 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5f86a8-54b4-4da3-b161-fc4fbebc59af None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:21.209331 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 538/1080] 10.222.0.102 () {56 vars in 1153 bytes} [Thu Sep 1 17:19:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:19:21.211457 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5f86a8-54b4-4da3-b161-fc4fbebc59af None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:21.238890 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86985c10-6f2d-4bd0-b1c5-071ffc332d89 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:21.240814 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86985c10-6f2d-4bd0-b1c5-071ffc332d89 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:21.299743 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:21.322864 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5f86a8-54b4-4da3-b161-fc4fbebc59af None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:21.350805 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 543/1081] 10.222.0.102 () {58 vars in 1287 bytes} [Thu Sep 1 17:19:20 2022] GET /identity/v3/auth/tokens => generated 2123 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 17:19:21.356439 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86985c10-6f2d-4bd0-b1c5-071ffc332d89 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:21.364232 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c20e1f-9da2-4cae-a245-f6a46175d9ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:21.365353 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c20e1f-9da2-4cae-a245-f6a46175d9ab None admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:21.365643 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c20e1f-9da2-4cae-a245-f6a46175d9ab None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:21.365918 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c20e1f-9da2-4cae-a245-f6a46175d9ab None admin] PATH_INFO: `/v3/projects/e78fff7f1f6e4f68b8d233ad2e5bbd4f/users/5f8293316baf4774b62fc11d55d50865/roles/9329b00eca174ca5a29094e84e243fa4` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:21.368794 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c20e1f-9da2-4cae-a245-f6a46175d9ab None admin] RBAC: Authorizing `identity:create_grant(project_id=e78fff7f1f6e4f68b8d233ad2e5bbd4f, user_id=5f8293316baf4774b62fc11d55d50865, role_id=9329b00eca174ca5a29094e84e243fa4)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:21.370439 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c20e1f-9da2-4cae-a245-f6a46175d9ab None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:21.385697 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 539/1082] 10.222.0.102 () {58 vars in 1444 bytes} [Thu Sep 1 17:19:21 2022] PUT /identity/v3/projects/e78fff7f1f6e4f68b8d233ad2e5bbd4f/users/5f8293316baf4774b62fc11d55d50865/roles/9329b00eca174ca5a29094e84e243fa4 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:19:21.473162 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86985c10-6f2d-4bd0-b1c5-071ffc332d89 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:21.513349 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8252caa1-098b-4499-90b9-ab99327ee2f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:21.514190 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8252caa1-098b-4499-90b9-ab99327ee2f8 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:21.514583 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8252caa1-098b-4499-90b9-ab99327ee2f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:21.514956 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8252caa1-098b-4499-90b9-ab99327ee2f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:21.528383 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8252caa1-098b-4499-90b9-ab99327ee2f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:21.529373 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8252caa1-098b-4499-90b9-ab99327ee2f8 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:21.546177 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-8252caa1-098b-4499-90b9-ab99327ee2f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:19:21.546177 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:19:21.546177 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:19:21.546177 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:19:21.546177 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:19:21.546177 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:19:21.546177 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:19:21.546177 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:19:21.546177 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:19:21.546177 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:19:21.546177 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:19:21.546177 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:19:21.546177 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:19:21.546177 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:19:21.546177 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:19:21.546177 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:19:21.547342 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:19:21.547342 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:19:21.547342 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:19:21.547342 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:19:21.547342 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:19:21.547342 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:19:21.547342 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:19:21.547342 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:19:21.547342 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:19:21.547342 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:19:21.547342 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:19:21.547342 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:19:21.547342 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:19:21.547342 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:19:21.547342 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:19:21.547342 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:19:21.547342 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:19:21.548439 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:19:21.548439 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:19:21.548439 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:19:21.548439 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 544/1083] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:19:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:19:21.553400 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c20e1f-9da2-4cae-a245-f6a46175d9ab None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:21.569978 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:21.661921 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c20e1f-9da2-4cae-a245-f6a46175d9ab None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:21.699670 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e730f7-8edd-43e6-bdc7-3bdf6524464c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:21.700282 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e730f7-8edd-43e6-bdc7-3bdf6524464c None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:21.700559 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e730f7-8edd-43e6-bdc7-3bdf6524464c None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:21.700831 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e730f7-8edd-43e6-bdc7-3bdf6524464c None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:21.701560 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e730f7-8edd-43e6-bdc7-3bdf6524464c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:19:21.701866 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e730f7-8edd-43e6-bdc7-3bdf6524464c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:21.703227 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e730f7-8edd-43e6-bdc7-3bdf6524464c None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:21.715073 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 540/1084] 10.222.0.102 () {56 vars in 1114 bytes} [Thu Sep 1 17:19:21 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:19:22.964882 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79a55b4-0e84-43ea-a42b-807c23fd1236 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:22.965484 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79a55b4-0e84-43ea-a42b-807c23fd1236 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:22.965484 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79a55b4-0e84-43ea-a42b-807c23fd1236 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:22.966823 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 545/1085] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:19:22 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:19:22.972181 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ccbece-f592-43cb-a04c-b15a6ca7326b None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:22.972603 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ccbece-f592-43cb-a04c-b15a6ca7326b None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:22.972831 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ccbece-f592-43cb-a04c-b15a6ca7326b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:22.991107 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-38ccbece-f592-43cb-a04c-b15a6ca7326b None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:19:22.992294 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-38ccbece-f592-43cb-a04c-b15a6ca7326b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:19:23.066250 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 541/1086] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:19:22 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:19:23.308732 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61302c5a-e05a-4d25-869f-90d1dc37f048 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:23.309131 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61302c5a-e05a-4d25-869f-90d1dc37f048 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:23.309452 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61302c5a-e05a-4d25-869f-90d1dc37f048 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:23.310807 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 546/1087] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:19:23 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:19:23.316732 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb7fc31-abb9-4b05-869f-b2d2f0dbbb5a None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:23.316964 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb7fc31-abb9-4b05-869f-b2d2f0dbbb5a None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:23.317418 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb7fc31-abb9-4b05-869f-b2d2f0dbbb5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:23.324113 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:23.336640 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-ceb7fc31-abb9-4b05-869f-b2d2f0dbbb5a None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:19:23.337832 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-ceb7fc31-abb9-4b05-869f-b2d2f0dbbb5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:19:23.349057 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61302c5a-e05a-4d25-869f-90d1dc37f048 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:23.362063 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:23.406997 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 542/1088] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:19:23 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:19:23.455548 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61302c5a-e05a-4d25-869f-90d1dc37f048 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:23.488912 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5ca04a-514e-453d-85db-4ea0238ea843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:23.489561 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5ca04a-514e-453d-85db-4ea0238ea843 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:23.489840 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5ca04a-514e-453d-85db-4ea0238ea843 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:23.490114 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5ca04a-514e-453d-85db-4ea0238ea843 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:23.499922 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5ca04a-514e-453d-85db-4ea0238ea843 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:19:23.504717 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-ce5ca04a-514e-453d-85db-4ea0238ea843 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:19:23.504717 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:19:23.504717 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:19:23.504717 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:19:23.504717 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:19:23.504717 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:19:23.504717 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:19:23.504717 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:19:23.504717 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:19:23.504717 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:19:23.504717 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:19:23.504717 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:19:23.504717 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:19:23.504717 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:19:23.504717 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:19:23.504717 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:19:23.505749 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:19:23.505749 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:19:23.505749 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:19:23.505749 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:19:23.505749 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:19:23.505749 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:19:23.505749 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:19:23.505749 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:19:23.505749 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:19:23.505749 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:19:23.505749 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:19:23.505749 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:19:23.505749 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:19:23.505749 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:19:23.505749 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:19:23.505749 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:19:23.506593 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:19:23.506593 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:19:23.506593 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:19:23.506593 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:19:23.506593 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:19:23.506593 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:19:23.506593 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:19:23.506593 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:19:23.506593 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:19:23.506593 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:19:23.506593 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:19:23.506593 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:19:23.506593 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 547/1089] 10.222.0.102 () {56 vars in 1101 bytes} [Thu Sep 1 17:19:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:19:23.511795 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb7fc31-abb9-4b05-869f-b2d2f0dbbb5a None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:23.611181 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb7fc31-abb9-4b05-869f-b2d2f0dbbb5a None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:23.639717 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:23.645855 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015b60d5-e6a0-4664-90d2-584107549ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:23.646379 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015b60d5-e6a0-4664-90d2-584107549ded None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:23.646646 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015b60d5-e6a0-4664-90d2-584107549ded None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:23.646912 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015b60d5-e6a0-4664-90d2-584107549ded None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:23.647568 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015b60d5-e6a0-4664-90d2-584107549ded None admin] RBAC: Adding query filter params (name=admin) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:19:23.647869 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015b60d5-e6a0-4664-90d2-584107549ded None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:23.649227 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015b60d5-e6a0-4664-90d2-584107549ded None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:23.657443 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 543/1090] 10.222.0.102 () {56 vars in 1104 bytes} [Thu Sep 1 17:19:23 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:19:23.663227 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5ca04a-514e-453d-85db-4ea0238ea843 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:23.677293 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015b60d5-e6a0-4664-90d2-584107549ded None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:23.736524 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:23.773861 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5ca04a-514e-453d-85db-4ea0238ea843 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:23.785937 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015b60d5-e6a0-4664-90d2-584107549ded None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:23.807382 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4919964-60ff-4c27-b36a-ea8e16b564da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:23.808017 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4919964-60ff-4c27-b36a-ea8e16b564da None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:23.808313 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4919964-60ff-4c27-b36a-ea8e16b564da None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:23.808563 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4919964-60ff-4c27-b36a-ea8e16b564da None admin] PATH_INFO: `/v3/users/3f85c02d50424a14a44a2eaa5e3c5fff` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:23.809735 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4919964-60ff-4c27-b36a-ea8e16b564da None admin] RBAC: Authorizing `identity:get_user(user_id=3f85c02d50424a14a44a2eaa5e3c5fff)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:23.811156 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4919964-60ff-4c27-b36a-ea8e16b564da None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:23.812743 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 548/1091] 10.222.0.102 () {56 vars in 1182 bytes} [Thu Sep 1 17:19:23 2022] GET /identity/v3/users/3f85c02d50424a14a44a2eaa5e3c5fff => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:19:23.818975 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4919964-60ff-4c27-b36a-ea8e16b564da None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:23.820733 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dd0bfc-67dc-440d-9c98-85385f002d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:23.821553 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dd0bfc-67dc-440d-9c98-85385f002d34 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:23.821848 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dd0bfc-67dc-440d-9c98-85385f002d34 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:23.822127 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dd0bfc-67dc-440d-9c98-85385f002d34 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:23.835334 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dd0bfc-67dc-440d-9c98-85385f002d34 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:19:23.840144 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-75dd0bfc-67dc-440d-9c98-85385f002d34 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 17:19:23.840144 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:19:23.840144 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:19:23.840144 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:19:23.840144 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:19:23.840144 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:19:23.840144 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:19:23.840144 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:19:23.840144 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:19:23.840144 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:19:23.840144 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:19:23.840144 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:19:23.840144 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:19:23.840144 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:19:23.840144 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:19:23.840144 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:19:23.841145 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:19:23.841145 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:19:23.841145 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:19:23.841145 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:19:23.841145 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:19:23.841145 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:19:23.841145 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:19:23.841145 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:19:23.841145 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:19:23.841145 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:19:23.841145 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:19:23.841145 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:19:23.841145 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:19:23.841145 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:19:23.841145 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:19:23.841145 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:19:23.842017 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:19:23.842017 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:19:23.842017 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:19:23.842017 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:19:23.842017 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:19:23.842017 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:19:23.842017 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:19:23.842017 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:19:23.842017 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:19:23.842017 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:19:23.842017 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 17:19:23.842017 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:19:23.842959 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 544/1092] 10.222.0.102 () {56 vars in 1125 bytes} [Thu Sep 1 17:19:23 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:19:23.847540 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dd0bfc-67dc-440d-9c98-85385f002d34 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:23.903217 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:23.920396 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4919964-60ff-4c27-b36a-ea8e16b564da None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:23.948844 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dd0bfc-67dc-440d-9c98-85385f002d34 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:23.954986 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87601261-4895-477c-9bd8-593210c0e772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:23.955636 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87601261-4895-477c-9bd8-593210c0e772 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:23.956024 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87601261-4895-477c-9bd8-593210c0e772 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:23.956299 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87601261-4895-477c-9bd8-593210c0e772 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:23.959932 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:23.988205 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bd26c8-269a-43e6-a529-60cfd1825cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:23.988822 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bd26c8-269a-43e6-a529-60cfd1825cd4 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:23.989236 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bd26c8-269a-43e6-a529-60cfd1825cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:23.989604 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bd26c8-269a-43e6-a529-60cfd1825cd4 None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:23.990276 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bd26c8-269a-43e6-a529-60cfd1825cd4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:19:23.990810 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bd26c8-269a-43e6-a529-60cfd1825cd4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:23.991965 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bd26c8-269a-43e6-a529-60cfd1825cd4 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:24.000425 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 545/1093] 10.222.0.102 () {56 vars in 1120 bytes} [Thu Sep 1 17:19:23 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 340 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:19:24.007977 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bd26c8-269a-43e6-a529-60cfd1825cd4 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:24.033098 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87601261-4895-477c-9bd8-593210c0e772 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:24.034531 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87601261-4895-477c-9bd8-593210c0e772 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:24.128688 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bd26c8-269a-43e6-a529-60cfd1825cd4 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:24.147545 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:24.153621 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 549/1094] 10.222.0.102 () {58 vars in 1287 bytes} [Thu Sep 1 17:19:23 2022] GET /identity/v3/auth/tokens => generated 2123 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 01 17:19:24.159562 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87601261-4895-477c-9bd8-593210c0e772 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:24.170821 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b83b5b0-4511-420a-b674-fcdd1a94d4de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:24.171457 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b83b5b0-4511-420a-b674-fcdd1a94d4de None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:24.171737 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b83b5b0-4511-420a-b674-fcdd1a94d4de None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:24.172029 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b83b5b0-4511-420a-b674-fcdd1a94d4de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:24.183106 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b83b5b0-4511-420a-b674-fcdd1a94d4de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:24.184606 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b83b5b0-4511-420a-b674-fcdd1a94d4de None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:24.194581 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:24.199225 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-7b83b5b0-4511-420a-b674-fcdd1a94d4de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:19:24.199225 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:19:24.199225 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:19:24.199225 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:19:24.199225 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:19:24.199225 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:19:24.199225 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:19:24.199225 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:19:24.199225 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:19:24.199225 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:19:24.199225 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:19:24.199225 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:19:24.199225 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:19:24.199225 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:19:24.199225 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:19:24.199225 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:19:24.200353 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:19:24.200353 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:19:24.200353 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:19:24.200353 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:19:24.200353 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:19:24.200353 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:19:24.200353 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:19:24.200353 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:19:24.200353 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:19:24.200353 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:19:24.200353 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:19:24.200353 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:19:24.200353 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:19:24.200353 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:19:24.200353 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:19:24.200353 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:19:24.200353 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:19:24.201462 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:19:24.201462 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:19:24.201462 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:19:24.201462 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 546/1095] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:19:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:19:24.206302 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b83b5b0-4511-420a-b674-fcdd1a94d4de None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:24.260257 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87601261-4895-477c-9bd8-593210c0e772 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:24.293037 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c7f45c-be0a-4ebc-8377-095d21dd1166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:24.293691 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c7f45c-be0a-4ebc-8377-095d21dd1166 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:24.293973 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c7f45c-be0a-4ebc-8377-095d21dd1166 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:24.294248 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c7f45c-be0a-4ebc-8377-095d21dd1166 None admin] PATH_INFO: `/v3/projects/8c162948f1a446698552d4407da48267` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:24.295361 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c7f45c-be0a-4ebc-8377-095d21dd1166 None admin] RBAC: Authorizing `identity:get_project(project_id=8c162948f1a446698552d4407da48267)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:24.296775 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c7f45c-be0a-4ebc-8377-095d21dd1166 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:24.298943 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 550/1096] 10.222.0.102 () {56 vars in 1191 bytes} [Thu Sep 1 17:19:24 2022] GET /identity/v3/projects/8c162948f1a446698552d4407da48267 => generated 312 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:19:24.304776 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b83b5b0-4511-420a-b674-fcdd1a94d4de None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:24.306128 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c7f45c-be0a-4ebc-8377-095d21dd1166 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:24.339274 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81ec63c-7d0d-48e6-a820-899bb74d81cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:24.339878 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81ec63c-7d0d-48e6-a820-899bb74d81cf None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:24.340161 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81ec63c-7d0d-48e6-a820-899bb74d81cf None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:24.340431 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81ec63c-7d0d-48e6-a820-899bb74d81cf None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:24.341058 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81ec63c-7d0d-48e6-a820-899bb74d81cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:19:24.341387 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81ec63c-7d0d-48e6-a820-899bb74d81cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:24.342762 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81ec63c-7d0d-48e6-a820-899bb74d81cf None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:24.355614 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 547/1097] 10.222.0.102 () {56 vars in 1114 bytes} [Thu Sep 1 17:19:24 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:19:24.361882 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81ec63c-7d0d-48e6-a820-899bb74d81cf None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:24.389846 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:24.436497 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c7f45c-be0a-4ebc-8377-095d21dd1166 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:24.439273 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:24.466444 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81ec63c-7d0d-48e6-a820-899bb74d81cf None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:24.471664 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0822a84-399f-4525-80f6-219ff7032812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:24.472265 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0822a84-399f-4525-80f6-219ff7032812 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:24.472544 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0822a84-399f-4525-80f6-219ff7032812 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:24.472818 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0822a84-399f-4525-80f6-219ff7032812 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:24.473596 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0822a84-399f-4525-80f6-219ff7032812 None admin] RBAC: Adding query filter params (role.id=36577a6f46d642d4b81805ede9872490, scope.project.id=8c162948f1a446698552d4407da48267, user.id=3f85c02d50424a14a44a2eaa5e3c5fff) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:19:24.473877 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0822a84-399f-4525-80f6-219ff7032812 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:24.475291 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0822a84-399f-4525-80f6-219ff7032812 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:24.481383 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 551/1098] 10.222.0.102 () {56 vars in 1379 bytes} [Thu Sep 1 17:19:24 2022] GET /identity/v3/role_assignments?user.id=3f85c02d50424a14a44a2eaa5e3c5fff&scope.project.id=8c162948f1a446698552d4407da48267&role.id=36577a6f46d642d4b81805ede9872490 => generated 610 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:19:24.504131 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2425a2d2-93f5-47d1-8e29-f42e41574b9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:24.504792 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2425a2d2-93f5-47d1-8e29-f42e41574b9b None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:24.505072 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2425a2d2-93f5-47d1-8e29-f42e41574b9b None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:24.505379 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2425a2d2-93f5-47d1-8e29-f42e41574b9b None admin] PATH_INFO: `/v3/users/nova` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:24.513975 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2425a2d2-93f5-47d1-8e29-f42e41574b9b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:24.515470 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2425a2d2-93f5-47d1-8e29-f42e41574b9b None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:24.528785 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-2425a2d2-93f5-47d1-8e29-f42e41574b9b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 17:19:24.528785 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:19:24.528785 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:19:24.528785 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:19:24.528785 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:19:24.528785 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:19:24.528785 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:19:24.528785 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:19:24.528785 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:19:24.528785 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:19:24.528785 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:19:24.528785 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:19:24.528785 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:19:24.528785 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:19:24.528785 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:19:24.528785 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:19:24.529862 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:19:24.529862 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:19:24.529862 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:19:24.529862 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:19:24.529862 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:19:24.529862 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:19:24.529862 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:19:24.529862 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:19:24.529862 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:19:24.529862 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:19:24.529862 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:19:24.529862 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:19:24.529862 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:19:24.529862 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:19:24.529862 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:19:24.529862 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:19:24.529862 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:19:24.529862 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:19:24.530810 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:19:24.530810 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:19:24.530810 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:19:24.530810 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:19:24.530810 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:19:24.530810 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:19:24.530810 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:19:24.530810 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:19:24.530810 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 17:19:24.530810 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:19:24.530810 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 548/1099] 10.222.0.102 () {56 vars in 1098 bytes} [Thu Sep 1 17:19:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:19:24.536409 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0822a84-399f-4525-80f6-219ff7032812 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:24.642351 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0822a84-399f-4525-80f6-219ff7032812 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:24.677105 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b06418-6e80-480e-831f-d2a346d9edca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:24.678257 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b06418-6e80-480e-831f-d2a346d9edca None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:24.678257 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b06418-6e80-480e-831f-d2a346d9edca None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:24.678482 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b06418-6e80-480e-831f-d2a346d9edca None admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:24.679034 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b06418-6e80-480e-831f-d2a346d9edca None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:19:24.679337 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b06418-6e80-480e-831f-d2a346d9edca None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:24.680765 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b06418-6e80-480e-831f-d2a346d9edca None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:24.698076 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 552/1100] 10.222.0.102 () {56 vars in 1138 bytes} [Thu Sep 1 17:19:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 370 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:19:24.704247 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2425a2d2-93f5-47d1-8e29-f42e41574b9b None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:24.793040 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:24.805818 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2425a2d2-93f5-47d1-8e29-f42e41574b9b None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:24.847463 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeab780e-ebc3-482b-adc6-59cc3c69b0f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:24.848273 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeab780e-ebc3-482b-adc6-59cc3c69b0f0 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:24.848626 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeab780e-ebc3-482b-adc6-59cc3c69b0f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:24.848982 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeab780e-ebc3-482b-adc6-59cc3c69b0f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:24.926612 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeab780e-ebc3-482b-adc6-59cc3c69b0f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:24.928070 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeab780e-ebc3-482b-adc6-59cc3c69b0f0 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:25.015563 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:25.029355 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 549/1101] 10.222.0.102 () {58 vars in 1287 bytes} [Thu Sep 1 17:19:24 2022] GET /identity/v3/auth/tokens => generated 2123 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 17:19:25.035013 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b06418-6e80-480e-831f-d2a346d9edca None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:25.062486 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:25.136157 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b06418-6e80-480e-831f-d2a346d9edca None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:25.170844 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93309f11-d6ae-41f0-b500-6c9b6d0eef10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:25.171467 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93309f11-d6ae-41f0-b500-6c9b6d0eef10 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:25.171745 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93309f11-d6ae-41f0-b500-6c9b6d0eef10 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:25.172018 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93309f11-d6ae-41f0-b500-6c9b6d0eef10 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:25.181851 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93309f11-d6ae-41f0-b500-6c9b6d0eef10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:25.183269 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93309f11-d6ae-41f0-b500-6c9b6d0eef10 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:25.198559 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-93309f11-d6ae-41f0-b500-6c9b6d0eef10 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:19:25.198559 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:19:25.198559 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:19:25.198559 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:19:25.198559 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:19:25.198559 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:19:25.198559 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:19:25.198559 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:19:25.198559 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:19:25.198559 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:19:25.198559 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:19:25.198559 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:19:25.198559 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:19:25.198559 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:19:25.198559 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:19:25.198559 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:19:25.199719 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:19:25.199719 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:19:25.199719 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:19:25.199719 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:19:25.199719 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:19:25.199719 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:19:25.199719 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:19:25.199719 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:19:25.199719 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:19:25.199719 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:19:25.199719 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:19:25.199719 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:19:25.199719 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:19:25.199719 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:19:25.199719 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:19:25.199719 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:19:25.199719 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:19:25.200906 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:19:25.200906 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:19:25.200906 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:19:25.201463 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 553/1102] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:19:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:19:25.205854 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeab780e-ebc3-482b-adc6-59cc3c69b0f0 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:25.304773 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeab780e-ebc3-482b-adc6-59cc3c69b0f0 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:25.338491 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bcbd59-27c3-474a-923a-67e269798460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:25.339088 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bcbd59-27c3-474a-923a-67e269798460 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:25.339362 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bcbd59-27c3-474a-923a-67e269798460 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:25.339628 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bcbd59-27c3-474a-923a-67e269798460 None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:25.340256 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bcbd59-27c3-474a-923a-67e269798460 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:19:25.340555 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bcbd59-27c3-474a-923a-67e269798460 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:25.342435 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bcbd59-27c3-474a-923a-67e269798460 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:25.354800 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 550/1103] 10.222.0.102 () {56 vars in 1114 bytes} [Thu Sep 1 17:19:25 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:19:25.360508 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93309f11-d6ae-41f0-b500-6c9b6d0eef10 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:25.436690 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:25.458871 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93309f11-d6ae-41f0-b500-6c9b6d0eef10 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:25.493231 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f758d284-7592-45ff-9181-165c8a848b10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:25.493889 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f758d284-7592-45ff-9181-165c8a848b10 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:25.494176 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f758d284-7592-45ff-9181-165c8a848b10 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:25.494449 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f758d284-7592-45ff-9181-165c8a848b10 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:25.501832 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f758d284-7592-45ff-9181-165c8a848b10 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:25.503259 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f758d284-7592-45ff-9181-165c8a848b10 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:25.514600 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-f758d284-7592-45ff-9181-165c8a848b10 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:19:25.514600 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:19:25.514600 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:19:25.514600 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:19:25.514600 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:19:25.514600 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:19:25.514600 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:19:25.514600 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:19:25.514600 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:19:25.514600 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:19:25.514600 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:19:25.514600 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:19:25.514600 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:19:25.514600 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:19:25.514600 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:19:25.521209 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:19:25.521209 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:19:25.521209 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:19:25.521209 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:19:25.521209 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:19:25.521209 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:19:25.521209 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:19:25.521209 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:19:25.521209 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:19:25.521209 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:19:25.521209 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:19:25.521209 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:19:25.521209 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:19:25.521209 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:19:25.521209 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:19:25.521209 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:19:25.521209 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:19:25.522200 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:19:25.522200 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:19:25.522200 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:19:25.522200 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:19:25.522200 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:19:25.522200 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:19:25.522200 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:19:25.522200 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:19:25.522200 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 554/1104] 10.222.0.102 () {56 vars in 1116 bytes} [Thu Sep 1 17:19:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:19:25.522690 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bcbd59-27c3-474a-923a-67e269798460 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:25.570914 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:25.640640 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bcbd59-27c3-474a-923a-67e269798460 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:25.675297 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b79d599-57b7-47af-8a90-c0fcab7d6db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:25.675898 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b79d599-57b7-47af-8a90-c0fcab7d6db4 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:25.676177 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b79d599-57b7-47af-8a90-c0fcab7d6db4 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:25.676452 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b79d599-57b7-47af-8a90-c0fcab7d6db4 None admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:25.677140 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b79d599-57b7-47af-8a90-c0fcab7d6db4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:19:25.677477 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b79d599-57b7-47af-8a90-c0fcab7d6db4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:25.678911 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b79d599-57b7-47af-8a90-c0fcab7d6db4 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:25.691844 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 551/1105] 10.222.0.102 () {56 vars in 1153 bytes} [Thu Sep 1 17:19:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:19:25.698015 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f758d284-7592-45ff-9181-165c8a848b10 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:25.805746 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f758d284-7592-45ff-9181-165c8a848b10 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:25.841078 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfe7fe0-8ebc-4f84-a1c4-4196c38a5eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:25.841641 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfe7fe0-8ebc-4f84-a1c4-4196c38a5eb8 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:25.841923 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfe7fe0-8ebc-4f84-a1c4-4196c38a5eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:25.842198 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfe7fe0-8ebc-4f84-a1c4-4196c38a5eb8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:25.842921 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfe7fe0-8ebc-4f84-a1c4-4196c38a5eb8 None admin] RBAC: Adding query filter params (role.id=9329b00eca174ca5a29094e84e243fa4, scope.project.id=e78fff7f1f6e4f68b8d233ad2e5bbd4f, user.id=5f8293316baf4774b62fc11d55d50865) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:19:25.843231 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfe7fe0-8ebc-4f84-a1c4-4196c38a5eb8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:25.845112 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfe7fe0-8ebc-4f84-a1c4-4196c38a5eb8 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:25.851406 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 555/1106] 10.222.0.102 () {56 vars in 1379 bytes} [Thu Sep 1 17:19:25 2022] GET /identity/v3/role_assignments?user.id=5f8293316baf4774b62fc11d55d50865&scope.project.id=e78fff7f1f6e4f68b8d233ad2e5bbd4f&role.id=9329b00eca174ca5a29094e84e243fa4 => generated 610 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:19:26.434383 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7cf059-f9ab-4bb9-a06b-57407e2070ab None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:26.434901 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7cf059-f9ab-4bb9-a06b-57407e2070ab None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:26.434990 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7cf059-f9ab-4bb9-a06b-57407e2070ab None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:26.436278 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 552/1107] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:19:26 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:19:26.442367 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab8efbb-f174-405a-b795-bb30d51d4f7d None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:26.442824 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab8efbb-f174-405a-b795-bb30d51d4f7d None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:26.442915 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab8efbb-f174-405a-b795-bb30d51d4f7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:26.449268 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:26.462297 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-5ab8efbb-f174-405a-b795-bb30d51d4f7d None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:19:26.463492 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-5ab8efbb-f174-405a-b795-bb30d51d4f7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:19:26.550276 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 556/1108] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:19:26 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:19:26.819916 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7cf059-f9ab-4bb9-a06b-57407e2070ab None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:26.946639 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7cf059-f9ab-4bb9-a06b-57407e2070ab None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:26.960322 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:26.987218 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c328c490-74a0-468d-a80a-85b2ee7e7669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:26.987840 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c328c490-74a0-468d-a80a-85b2ee7e7669 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:26.988116 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c328c490-74a0-468d-a80a-85b2ee7e7669 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:26.988388 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c328c490-74a0-468d-a80a-85b2ee7e7669 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:26.989513 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c328c490-74a0-468d-a80a-85b2ee7e7669 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:26.990935 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c328c490-74a0-468d-a80a-85b2ee7e7669 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:26.992549 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 553/1109] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:19:26 2022] GET /identity/v3/domains/default => generated 200 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:19:26.998380 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab8efbb-f174-405a-b795-bb30d51d4f7d None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:27.095297 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab8efbb-f174-405a-b795-bb30d51d4f7d None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:27.123272 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:27.128621 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db0313c-a202-42b6-9c2a-ba86dc19926f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:27.129219 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db0313c-a202-42b6-9c2a-ba86dc19926f None admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:27.129545 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db0313c-a202-42b6-9c2a-ba86dc19926f None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:27.129826 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db0313c-a202-42b6-9c2a-ba86dc19926f None admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:27.130464 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db0313c-a202-42b6-9c2a-ba86dc19926f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:27.131885 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db0313c-a202-42b6-9c2a-ba86dc19926f None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:27.156578 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 557/1110] 10.222.0.102 () {60 vars in 1137 bytes} [Thu Sep 1 17:19:26 2022] POST /identity/v3/users => generated 287 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:19:29.082546 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e28c3d-85df-4267-b616-147467561181 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:29.083049 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e28c3d-85df-4267-b616-147467561181 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:29.083049 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e28c3d-85df-4267-b616-147467561181 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:29.084335 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 554/1111] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:19:29 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:19:29.089441 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0426bf12-cb05-4c16-943d-a9d298ce68e4 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:29.089846 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0426bf12-cb05-4c16-943d-a9d298ce68e4 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:29.089982 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0426bf12-cb05-4c16-943d-a9d298ce68e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:29.107996 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-0426bf12-cb05-4c16-943d-a9d298ce68e4 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:19:29.109164 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-0426bf12-cb05-4c16-943d-a9d298ce68e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:19:29.174377 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 558/1112] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:19:29 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:19:29.446003 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e28c3d-85df-4267-b616-147467561181 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:29.545614 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e28c3d-85df-4267-b616-147467561181 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:29.579469 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d5be5c-ca0e-4153-90df-a91a985dc61f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:29.580093 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d5be5c-ca0e-4153-90df-a91a985dc61f None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:29.580374 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d5be5c-ca0e-4153-90df-a91a985dc61f None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:29.580649 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d5be5c-ca0e-4153-90df-a91a985dc61f None admin] PATH_INFO: `/v3/roles/abef78c185c74c16a2c7e8a170583f9f` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:29.582066 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d5be5c-ca0e-4153-90df-a91a985dc61f None admin] RBAC: Authorizing `identity:get_role(role_id=abef78c185c74c16a2c7e8a170583f9f)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:29.583440 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d5be5c-ca0e-4153-90df-a91a985dc61f None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:29.584774 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 555/1113] 10.222.0.102 () {56 vars in 1182 bytes} [Thu Sep 1 17:19:29 2022] GET /identity/v3/roles/abef78c185c74c16a2c7e8a170583f9f => generated 224 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:19:29.590618 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0426bf12-cb05-4c16-943d-a9d298ce68e4 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:29.686964 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0426bf12-cb05-4c16-943d-a9d298ce68e4 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:29.703696 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:29.721878 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b39373-1941-47d3-b553-cbc8ab84046e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:29.722503 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b39373-1941-47d3-b553-cbc8ab84046e None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:29.722785 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b39373-1941-47d3-b553-cbc8ab84046e None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:29.723060 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b39373-1941-47d3-b553-cbc8ab84046e None admin] PATH_INFO: `/v3/users/2d09cccd5ff64c9e8d6d1404c2d0bf92` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:29.736550 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b39373-1941-47d3-b553-cbc8ab84046e None admin] RBAC: Authorizing `identity:get_user(user_id=2d09cccd5ff64c9e8d6d1404c2d0bf92)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:29.738019 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b39373-1941-47d3-b553-cbc8ab84046e None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:29.739620 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 559/1114] 10.222.0.102 () {56 vars in 1182 bytes} [Thu Sep 1 17:19:29 2022] GET /identity/v3/users/2d09cccd5ff64c9e8d6d1404c2d0bf92 => generated 287 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:19:29.745880 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d5be5c-ca0e-4153-90df-a91a985dc61f None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:29.791062 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:29.843075 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d5be5c-ca0e-4153-90df-a91a985dc61f None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:29.877110 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca0bdca-aab5-4d9e-84f9-150af46fc72a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:29.877734 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca0bdca-aab5-4d9e-84f9-150af46fc72a None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:29.878015 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca0bdca-aab5-4d9e-84f9-150af46fc72a None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:29.878287 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca0bdca-aab5-4d9e-84f9-150af46fc72a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:29.942742 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca0bdca-aab5-4d9e-84f9-150af46fc72a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:29.944187 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca0bdca-aab5-4d9e-84f9-150af46fc72a None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:29.986953 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:30.041938 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 556/1115] 10.222.0.102 () {58 vars in 1287 bytes} [Thu Sep 1 17:19:29 2022] GET /identity/v3/auth/tokens => generated 2123 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 17:19:30.048423 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b39373-1941-47d3-b553-cbc8ab84046e None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:30.145384 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b39373-1941-47d3-b553-cbc8ab84046e None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:30.178180 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b583544-671b-4fd4-8a38-37ce219bc488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:30.178797 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b583544-671b-4fd4-8a38-37ce219bc488 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:30.179075 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b583544-671b-4fd4-8a38-37ce219bc488 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:30.179345 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b583544-671b-4fd4-8a38-37ce219bc488 None admin] PATH_INFO: `/v3/projects/8c162948f1a446698552d4407da48267` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:30.180478 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b583544-671b-4fd4-8a38-37ce219bc488 None admin] RBAC: Authorizing `identity:get_project(project_id=8c162948f1a446698552d4407da48267)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:30.181919 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b583544-671b-4fd4-8a38-37ce219bc488 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:30.183532 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 560/1116] 10.222.0.102 () {56 vars in 1191 bytes} [Thu Sep 1 17:19:30 2022] GET /identity/v3/projects/8c162948f1a446698552d4407da48267 => generated 312 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:19:30.189763 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca0bdca-aab5-4d9e-84f9-150af46fc72a None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:30.290450 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca0bdca-aab5-4d9e-84f9-150af46fc72a None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:30.323835 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef389bdd-e87d-4f6c-9001-7612ea9e628e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:30.324426 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef389bdd-e87d-4f6c-9001-7612ea9e628e None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:30.324701 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef389bdd-e87d-4f6c-9001-7612ea9e628e None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:30.324967 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef389bdd-e87d-4f6c-9001-7612ea9e628e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:30.325695 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef389bdd-e87d-4f6c-9001-7612ea9e628e None admin] RBAC: Adding query filter params (role.id=abef78c185c74c16a2c7e8a170583f9f, scope.project.id=8c162948f1a446698552d4407da48267, user.id=2d09cccd5ff64c9e8d6d1404c2d0bf92) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:19:30.325999 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef389bdd-e87d-4f6c-9001-7612ea9e628e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:30.327411 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef389bdd-e87d-4f6c-9001-7612ea9e628e None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:30.333213 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 557/1117] 10.222.0.102 () {56 vars in 1379 bytes} [Thu Sep 1 17:19:30 2022] GET /identity/v3/role_assignments?user.id=2d09cccd5ff64c9e8d6d1404c2d0bf92&scope.project.id=8c162948f1a446698552d4407da48267&role.id=abef78c185c74c16a2c7e8a170583f9f => generated 260 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:19:32.207253 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e02e1f-5d75-4251-a6d6-f47a59757e5d None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:32.209210 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e02e1f-5d75-4251-a6d6-f47a59757e5d None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:32.209210 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e02e1f-5d75-4251-a6d6-f47a59757e5d None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:32.209210 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 561/1118] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:19:32 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:19:32.214552 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d62f22e-9bf0-45b6-8f55-d7dc00d9fdac None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:32.215061 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d62f22e-9bf0-45b6-8f55-d7dc00d9fdac None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:32.215193 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d62f22e-9bf0-45b6-8f55-d7dc00d9fdac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:32.233509 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-7d62f22e-9bf0-45b6-8f55-d7dc00d9fdac None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:19:32.234680 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-7d62f22e-9bf0-45b6-8f55-d7dc00d9fdac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:19:32.237236 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:32.302024 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 558/1119] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:19:32 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:19:32.574225 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e02e1f-5d75-4251-a6d6-f47a59757e5d None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:32.630184 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:32.684703 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e02e1f-5d75-4251-a6d6-f47a59757e5d None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:32.716956 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4455b1-bbbb-4704-a241-701851be2c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:32.717606 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4455b1-bbbb-4704-a241-701851be2c95 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:32.717885 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4455b1-bbbb-4704-a241-701851be2c95 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:32.718156 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4455b1-bbbb-4704-a241-701851be2c95 None admin] PATH_INFO: `/v3/roles/abef78c185c74c16a2c7e8a170583f9f` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:32.720063 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4455b1-bbbb-4704-a241-701851be2c95 None admin] RBAC: Authorizing `identity:get_role(role_id=abef78c185c74c16a2c7e8a170583f9f)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:32.721452 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4455b1-bbbb-4704-a241-701851be2c95 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:32.722765 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 562/1120] 10.222.0.102 () {56 vars in 1182 bytes} [Thu Sep 1 17:19:32 2022] GET /identity/v3/roles/abef78c185c74c16a2c7e8a170583f9f => generated 224 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:19:32.728870 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d62f22e-9bf0-45b6-8f55-d7dc00d9fdac None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:32.823677 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d62f22e-9bf0-45b6-8f55-d7dc00d9fdac None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:32.856996 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2c326b-b352-428d-b899-dc1ae2f01233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:32.857588 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2c326b-b352-428d-b899-dc1ae2f01233 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:32.857875 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2c326b-b352-428d-b899-dc1ae2f01233 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:32.858151 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2c326b-b352-428d-b899-dc1ae2f01233 None admin] PATH_INFO: `/v3/users/2d09cccd5ff64c9e8d6d1404c2d0bf92` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:32.859265 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2c326b-b352-428d-b899-dc1ae2f01233 None admin] RBAC: Authorizing `identity:get_user(user_id=2d09cccd5ff64c9e8d6d1404c2d0bf92)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:32.861320 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2c326b-b352-428d-b899-dc1ae2f01233 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:32.862910 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 559/1121] 10.222.0.102 () {56 vars in 1182 bytes} [Thu Sep 1 17:19:32 2022] GET /identity/v3/users/2d09cccd5ff64c9e8d6d1404c2d0bf92 => generated 287 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:19:32.868621 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4455b1-bbbb-4704-a241-701851be2c95 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:32.965364 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4455b1-bbbb-4704-a241-701851be2c95 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:32.978230 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:32.999345 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d701c9-b09e-47e8-9e9d-92ff8647ac6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:32.999980 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d701c9-b09e-47e8-9e9d-92ff8647ac6c None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:33.000236 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d701c9-b09e-47e8-9e9d-92ff8647ac6c None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:33.000574 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d701c9-b09e-47e8-9e9d-92ff8647ac6c None admin] PATH_INFO: `/v3/projects/8c162948f1a446698552d4407da48267` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:33.001718 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d701c9-b09e-47e8-9e9d-92ff8647ac6c None admin] RBAC: Authorizing `identity:get_project(project_id=8c162948f1a446698552d4407da48267)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:33.003151 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d701c9-b09e-47e8-9e9d-92ff8647ac6c None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:33.004769 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 563/1122] 10.222.0.102 () {56 vars in 1191 bytes} [Thu Sep 1 17:19:32 2022] GET /identity/v3/projects/8c162948f1a446698552d4407da48267 => generated 312 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:19:33.010753 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2c326b-b352-428d-b899-dc1ae2f01233 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:33.019963 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:33.111638 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2c326b-b352-428d-b899-dc1ae2f01233 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:33.144618 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856b29a0-2957-4a66-8023-594d15cf64e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:33.145345 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856b29a0-2957-4a66-8023-594d15cf64e1 None admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:33.145633 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856b29a0-2957-4a66-8023-594d15cf64e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:33.145912 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856b29a0-2957-4a66-8023-594d15cf64e1 None admin] PATH_INFO: `/v3/projects/8c162948f1a446698552d4407da48267/users/2d09cccd5ff64c9e8d6d1404c2d0bf92/roles/abef78c185c74c16a2c7e8a170583f9f` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:33.148051 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856b29a0-2957-4a66-8023-594d15cf64e1 None admin] RBAC: Authorizing `identity:create_grant(project_id=8c162948f1a446698552d4407da48267, user_id=2d09cccd5ff64c9e8d6d1404c2d0bf92, role_id=abef78c185c74c16a2c7e8a170583f9f)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:33.149622 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856b29a0-2957-4a66-8023-594d15cf64e1 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:33.163058 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 560/1123] 10.222.0.102 () {58 vars in 1444 bytes} [Thu Sep 1 17:19:33 2022] PUT /identity/v3/projects/8c162948f1a446698552d4407da48267/users/2d09cccd5ff64c9e8d6d1404c2d0bf92/roles/abef78c185c74c16a2c7e8a170583f9f => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:19:35.023583 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791dd339-f0af-4afe-9e44-bfbaba045e50 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:35.024137 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791dd339-f0af-4afe-9e44-bfbaba045e50 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:35.024137 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791dd339-f0af-4afe-9e44-bfbaba045e50 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:35.025532 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 564/1124] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:19:35 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:19:35.030837 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a376fc-282b-4789-b3f8-f68e94bae71c None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:35.031257 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a376fc-282b-4789-b3f8-f68e94bae71c None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:35.031370 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a376fc-282b-4789-b3f8-f68e94bae71c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:35.050354 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-e4a376fc-282b-4789-b3f8-f68e94bae71c None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:19:35.051522 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-e4a376fc-282b-4789-b3f8-f68e94bae71c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:19:35.109032 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:35.117098 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 561/1125] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:19:35 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 17:19:35.390218 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791dd339-f0af-4afe-9e44-bfbaba045e50 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:35.490274 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791dd339-f0af-4afe-9e44-bfbaba045e50 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:35.523939 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c9484c-389d-403d-a5df-3af0436d2159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:35.524557 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c9484c-389d-403d-a5df-3af0436d2159 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:35.524837 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c9484c-389d-403d-a5df-3af0436d2159 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:35.525111 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c9484c-389d-403d-a5df-3af0436d2159 None admin] PATH_INFO: `/v3/roles/abef78c185c74c16a2c7e8a170583f9f` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:35.526535 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c9484c-389d-403d-a5df-3af0436d2159 None admin] RBAC: Authorizing `identity:get_role(role_id=abef78c185c74c16a2c7e8a170583f9f)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:35.527900 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c9484c-389d-403d-a5df-3af0436d2159 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:35.529213 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 565/1126] 10.222.0.102 () {56 vars in 1182 bytes} [Thu Sep 1 17:19:35 2022] GET /identity/v3/roles/abef78c185c74c16a2c7e8a170583f9f => generated 224 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:19:35.535756 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a376fc-282b-4789-b3f8-f68e94bae71c None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:35.632320 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a376fc-282b-4789-b3f8-f68e94bae71c None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:35.664734 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5373e189-aecc-4a30-9453-b46a98b719af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:35.665293 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5373e189-aecc-4a30-9453-b46a98b719af None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:35.665601 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5373e189-aecc-4a30-9453-b46a98b719af None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:35.665873 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5373e189-aecc-4a30-9453-b46a98b719af None admin] PATH_INFO: `/v3/users/2d09cccd5ff64c9e8d6d1404c2d0bf92` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:35.666975 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5373e189-aecc-4a30-9453-b46a98b719af None admin] RBAC: Authorizing `identity:get_user(user_id=2d09cccd5ff64c9e8d6d1404c2d0bf92)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:35.668401 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5373e189-aecc-4a30-9453-b46a98b719af None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:35.669990 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 562/1127] 10.222.0.102 () {56 vars in 1182 bytes} [Thu Sep 1 17:19:35 2022] GET /identity/v3/users/2d09cccd5ff64c9e8d6d1404c2d0bf92 => generated 287 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:19:35.676308 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c9484c-389d-403d-a5df-3af0436d2159 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:35.723434 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:35.772505 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c9484c-389d-403d-a5df-3af0436d2159 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:35.805348 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d52c63-7931-4270-8538-7c2d95b55ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:35.805942 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d52c63-7931-4270-8538-7c2d95b55ede None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:35.806221 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d52c63-7931-4270-8538-7c2d95b55ede None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:35.806494 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d52c63-7931-4270-8538-7c2d95b55ede None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:35.871969 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d52c63-7931-4270-8538-7c2d95b55ede None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:35.873427 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d52c63-7931-4270-8538-7c2d95b55ede None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:35.918800 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:35.969834 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 566/1128] 10.222.0.102 () {58 vars in 1287 bytes} [Thu Sep 1 17:19:35 2022] GET /identity/v3/auth/tokens => generated 2123 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 17:19:35.975560 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5373e189-aecc-4a30-9453-b46a98b719af None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:36.036281 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:36.074611 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5373e189-aecc-4a30-9453-b46a98b719af None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:36.115177 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e86593f-9696-4e18-b09a-7fe06652248e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:36.115790 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e86593f-9696-4e18-b09a-7fe06652248e None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:36.116066 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e86593f-9696-4e18-b09a-7fe06652248e None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:36.116335 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e86593f-9696-4e18-b09a-7fe06652248e None admin] PATH_INFO: `/v3/projects/8c162948f1a446698552d4407da48267` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:36.117446 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e86593f-9696-4e18-b09a-7fe06652248e None admin] RBAC: Authorizing `identity:get_project(project_id=8c162948f1a446698552d4407da48267)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:36.118862 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e86593f-9696-4e18-b09a-7fe06652248e None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:36.120474 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 563/1129] 10.222.0.102 () {56 vars in 1191 bytes} [Thu Sep 1 17:19:35 2022] GET /identity/v3/projects/8c162948f1a446698552d4407da48267 => generated 312 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:19:36.126710 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d52c63-7931-4270-8538-7c2d95b55ede None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:36.223581 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d52c63-7931-4270-8538-7c2d95b55ede None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:36.256831 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90d574e-90eb-41c4-a84c-961ca4126507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:36.257471 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90d574e-90eb-41c4-a84c-961ca4126507 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:36.257753 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90d574e-90eb-41c4-a84c-961ca4126507 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:36.258024 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90d574e-90eb-41c4-a84c-961ca4126507 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:36.258738 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90d574e-90eb-41c4-a84c-961ca4126507 None admin] RBAC: Adding query filter params (role.id=abef78c185c74c16a2c7e8a170583f9f, scope.project.id=8c162948f1a446698552d4407da48267, user.id=2d09cccd5ff64c9e8d6d1404c2d0bf92) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:19:36.259047 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90d574e-90eb-41c4-a84c-961ca4126507 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:36.260453 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90d574e-90eb-41c4-a84c-961ca4126507 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:36.266619 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 567/1130] 10.222.0.102 () {56 vars in 1379 bytes} [Thu Sep 1 17:19:36 2022] GET /identity/v3/role_assignments?user.id=2d09cccd5ff64c9e8d6d1404c2d0bf92&scope.project.id=8c162948f1a446698552d4407da48267&role.id=abef78c185c74c16a2c7e8a170583f9f => generated 610 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:19:38.178392 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76005ec-a3dd-4db3-8cbf-758243a125d4 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:38.178895 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76005ec-a3dd-4db3-8cbf-758243a125d4 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:38.178895 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76005ec-a3dd-4db3-8cbf-758243a125d4 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:38.180190 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 564/1131] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:19:38 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:19:38.185369 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d85251e-325d-44c0-ad90-0f5bc970fcc9 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:38.185789 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d85251e-325d-44c0-ad90-0f5bc970fcc9 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:38.185922 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d85251e-325d-44c0-ad90-0f5bc970fcc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:38.204799 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-9d85251e-325d-44c0-ad90-0f5bc970fcc9 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:19:38.206011 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-9d85251e-325d-44c0-ad90-0f5bc970fcc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:19:38.213336 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:38.274546 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 568/1132] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:19:38 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:19:38.552639 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76005ec-a3dd-4db3-8cbf-758243a125d4 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:38.652293 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76005ec-a3dd-4db3-8cbf-758243a125d4 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:38.679936 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:38.685343 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a5ec43-ec21-4ce5-8e43-1c665c3a0971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:38.685993 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a5ec43-ec21-4ce5-8e43-1c665c3a0971 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:38.686269 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a5ec43-ec21-4ce5-8e43-1c665c3a0971 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:38.686537 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a5ec43-ec21-4ce5-8e43-1c665c3a0971 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:38.688160 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a5ec43-ec21-4ce5-8e43-1c665c3a0971 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:38.689604 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a5ec43-ec21-4ce5-8e43-1c665c3a0971 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:38.691159 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 565/1133] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:19:38 2022] GET /identity/v3/domains/default => generated 200 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:19:38.697107 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d85251e-325d-44c0-ad90-0f5bc970fcc9 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:38.794920 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d85251e-325d-44c0-ad90-0f5bc970fcc9 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:38.828206 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3798ba95-6951-4264-8697-690eb703f9f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:38.829364 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3798ba95-6951-4264-8697-690eb703f9f6 None admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:38.829649 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3798ba95-6951-4264-8697-690eb703f9f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:38.829920 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3798ba95-6951-4264-8697-690eb703f9f6 None admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:38.830568 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3798ba95-6951-4264-8697-690eb703f9f6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:38.831996 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3798ba95-6951-4264-8697-690eb703f9f6 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:38.845831 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:38.860835 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 569/1134] 10.222.0.102 () {60 vars in 1146 bytes} [Thu Sep 1 17:19:38 2022] POST /identity/v3/projects => generated 312 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:19:40.856928 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303f9243-8f54-48e2-8ca5-d14ec1c7d659 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:40.857428 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303f9243-8f54-48e2-8ca5-d14ec1c7d659 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:40.857539 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303f9243-8f54-48e2-8ca5-d14ec1c7d659 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:40.858868 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 566/1135] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:19:40 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:19:40.864427 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c695d34-72a3-491c-9a8f-2ac141822baa None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:40.864863 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c695d34-72a3-491c-9a8f-2ac141822baa None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:40.864961 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c695d34-72a3-491c-9a8f-2ac141822baa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:40.884243 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-6c695d34-72a3-491c-9a8f-2ac141822baa None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:19:40.885458 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-6c695d34-72a3-491c-9a8f-2ac141822baa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:19:40.950304 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 570/1136] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:19:40 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:19:41.220252 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303f9243-8f54-48e2-8ca5-d14ec1c7d659 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:41.319817 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303f9243-8f54-48e2-8ca5-d14ec1c7d659 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:41.354222 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f251925-1e29-408f-9f59-a2909c1e3e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:41.354841 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f251925-1e29-408f-9f59-a2909c1e3e93 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:41.355120 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f251925-1e29-408f-9f59-a2909c1e3e93 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:41.355394 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f251925-1e29-408f-9f59-a2909c1e3e93 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:41.356491 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f251925-1e29-408f-9f59-a2909c1e3e93 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:41.357952 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f251925-1e29-408f-9f59-a2909c1e3e93 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:41.360048 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 567/1137] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:19:41 2022] GET /identity/v3/domains/default => generated 200 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:19:41.365835 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c695d34-72a3-491c-9a8f-2ac141822baa None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:41.462913 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c695d34-72a3-491c-9a8f-2ac141822baa None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:41.486452 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:41.497889 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27123d2-4727-4bd4-84ff-59d0634840de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:41.498495 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27123d2-4727-4bd4-84ff-59d0634840de None admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:41.498777 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27123d2-4727-4bd4-84ff-59d0634840de None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:41.499053 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27123d2-4727-4bd4-84ff-59d0634840de None admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:41.499703 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27123d2-4727-4bd4-84ff-59d0634840de None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:41.501814 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27123d2-4727-4bd4-84ff-59d0634840de None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:41.522051 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 571/1138] 10.222.0.102 () {60 vars in 1137 bytes} [Thu Sep 1 17:19:41 2022] POST /identity/v3/users => generated 287 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:19:43.442317 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4186521-27c8-435e-a331-f5e2c2dbd7ff None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:43.442850 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4186521-27c8-435e-a331-f5e2c2dbd7ff None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:43.442850 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4186521-27c8-435e-a331-f5e2c2dbd7ff None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:43.444201 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 568/1139] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:19:43 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:19:43.449266 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0052e34b-e6c1-4779-9967-69c4d746cf98 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:43.449668 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0052e34b-e6c1-4779-9967-69c4d746cf98 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:43.449869 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0052e34b-e6c1-4779-9967-69c4d746cf98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:43.467888 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-0052e34b-e6c1-4779-9967-69c4d746cf98 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:19:43.469598 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-0052e34b-e6c1-4779-9967-69c4d746cf98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:19:43.535102 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 572/1140] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:19:43 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:19:43.799551 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4186521-27c8-435e-a331-f5e2c2dbd7ff None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:43.867817 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:43.902501 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4186521-27c8-435e-a331-f5e2c2dbd7ff None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:43.935357 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643c829c-6611-41de-993c-361237e1736e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:43.935980 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643c829c-6611-41de-993c-361237e1736e None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:43.936265 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643c829c-6611-41de-993c-361237e1736e None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:43.936540 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643c829c-6611-41de-993c-361237e1736e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:43.947246 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643c829c-6611-41de-993c-361237e1736e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:19:43.952051 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-643c829c-6611-41de-993c-361237e1736e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:19:43.952051 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:19:43.952051 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:19:43.952051 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:19:43.952051 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:19:43.952051 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:19:43.952051 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:19:43.952051 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:19:43.952051 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:19:43.952051 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:19:43.952051 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:19:43.952051 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:19:43.952051 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:19:43.952051 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:19:43.952051 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:19:43.952051 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:19:43.953858 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:19:43.953858 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:19:43.953858 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:19:43.953858 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:19:43.953858 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:19:43.953858 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:19:43.953858 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:19:43.953858 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:19:43.953858 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:19:43.953858 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:19:43.953858 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:19:43.953858 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:19:43.953858 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:19:43.953858 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:19:43.953858 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:19:43.953858 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:19:43.954767 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:19:43.954767 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:19:43.954767 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:19:43.954767 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:19:43.954767 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:19:43.954767 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:19:43.954767 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:19:43.954767 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:19:43.954767 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:19:43.954767 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:19:43.954767 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:19:43.954767 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:19:43.954767 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 569/1141] 10.222.0.102 () {56 vars in 1101 bytes} [Thu Sep 1 17:19:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:19:43.959748 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0052e34b-e6c1-4779-9967-69c4d746cf98 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:44.056118 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0052e34b-e6c1-4779-9967-69c4d746cf98 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:44.067104 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:44.091360 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10165f9-3fea-4a32-8ca4-91e7d3a0a6f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:44.091962 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10165f9-3fea-4a32-8ca4-91e7d3a0a6f7 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:44.092246 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10165f9-3fea-4a32-8ca4-91e7d3a0a6f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:44.092522 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10165f9-3fea-4a32-8ca4-91e7d3a0a6f7 None admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:44.093181 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10165f9-3fea-4a32-8ca4-91e7d3a0a6f7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:19:44.093547 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10165f9-3fea-4a32-8ca4-91e7d3a0a6f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:44.094906 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10165f9-3fea-4a32-8ca4-91e7d3a0a6f7 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:44.102948 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 573/1142] 10.222.0.102 () {56 vars in 1104 bytes} [Thu Sep 1 17:19:43 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:19:44.108711 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643c829c-6611-41de-993c-361237e1736e None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:44.217761 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643c829c-6611-41de-993c-361237e1736e None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:44.244623 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:44.253184 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8e7d64-93e2-4a4d-ac20-61e16fad7c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:44.253917 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8e7d64-93e2-4a4d-ac20-61e16fad7c0a None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:44.254197 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8e7d64-93e2-4a4d-ac20-61e16fad7c0a None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:44.254471 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8e7d64-93e2-4a4d-ac20-61e16fad7c0a None admin] PATH_INFO: `/v3/users/07b366c5b4094404b65dac9eed5dc6fd` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:44.268020 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8e7d64-93e2-4a4d-ac20-61e16fad7c0a None admin] RBAC: Authorizing `identity:get_user(user_id=07b366c5b4094404b65dac9eed5dc6fd)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:44.270000 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8e7d64-93e2-4a4d-ac20-61e16fad7c0a None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:44.271117 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 570/1143] 10.222.0.102 () {56 vars in 1182 bytes} [Thu Sep 1 17:19:44 2022] GET /identity/v3/users/07b366c5b4094404b65dac9eed5dc6fd => generated 287 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:19:44.277049 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10165f9-3fea-4a32-8ca4-91e7d3a0a6f7 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:44.372977 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10165f9-3fea-4a32-8ca4-91e7d3a0a6f7 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:44.405215 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1213594-4e76-41ea-8b09-1ae32974cb68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:44.405834 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1213594-4e76-41ea-8b09-1ae32974cb68 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:44.406654 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1213594-4e76-41ea-8b09-1ae32974cb68 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:44.406923 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1213594-4e76-41ea-8b09-1ae32974cb68 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:44.450115 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:44.477116 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1213594-4e76-41ea-8b09-1ae32974cb68 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:44.478594 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1213594-4e76-41ea-8b09-1ae32974cb68 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:44.573168 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 574/1144] 10.222.0.102 () {58 vars in 1287 bytes} [Thu Sep 1 17:19:44 2022] GET /identity/v3/auth/tokens => generated 2123 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 17:19:44.579156 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8e7d64-93e2-4a4d-ac20-61e16fad7c0a None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:44.675871 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8e7d64-93e2-4a4d-ac20-61e16fad7c0a None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:44.708548 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a364e69d-d6b1-4fd2-b67f-af55a51afaf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:44.709168 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a364e69d-d6b1-4fd2-b67f-af55a51afaf3 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:44.709472 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a364e69d-d6b1-4fd2-b67f-af55a51afaf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:44.709743 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a364e69d-d6b1-4fd2-b67f-af55a51afaf3 None admin] PATH_INFO: `/v3/projects/e6d239c17ad14a2aba9cacdaa7fa25f3` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:44.710847 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a364e69d-d6b1-4fd2-b67f-af55a51afaf3 None admin] RBAC: Authorizing `identity:get_project(project_id=e6d239c17ad14a2aba9cacdaa7fa25f3)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:44.712273 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a364e69d-d6b1-4fd2-b67f-af55a51afaf3 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:44.713911 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 571/1145] 10.222.0.102 () {56 vars in 1191 bytes} [Thu Sep 1 17:19:44 2022] GET /identity/v3/projects/e6d239c17ad14a2aba9cacdaa7fa25f3 => generated 312 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:19:44.720546 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1213594-4e76-41ea-8b09-1ae32974cb68 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:44.788755 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:44.818543 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1213594-4e76-41ea-8b09-1ae32974cb68 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:44.851511 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa092e1-7e48-41fd-9fd8-4abe12b8829a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:44.852109 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa092e1-7e48-41fd-9fd8-4abe12b8829a None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:44.852384 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa092e1-7e48-41fd-9fd8-4abe12b8829a None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:44.852647 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa092e1-7e48-41fd-9fd8-4abe12b8829a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:44.853379 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa092e1-7e48-41fd-9fd8-4abe12b8829a None admin] RBAC: Adding query filter params (role.id=36577a6f46d642d4b81805ede9872490, scope.project.id=e6d239c17ad14a2aba9cacdaa7fa25f3, user.id=07b366c5b4094404b65dac9eed5dc6fd) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:19:44.853683 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa092e1-7e48-41fd-9fd8-4abe12b8829a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:44.855077 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa092e1-7e48-41fd-9fd8-4abe12b8829a None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:44.860511 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 575/1146] 10.222.0.102 () {56 vars in 1379 bytes} [Thu Sep 1 17:19:44 2022] GET /identity/v3/role_assignments?user.id=07b366c5b4094404b65dac9eed5dc6fd&scope.project.id=e6d239c17ad14a2aba9cacdaa7fa25f3&role.id=36577a6f46d642d4b81805ede9872490 => generated 260 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:19:46.706966 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adca707-ac43-4ecd-8b79-24311a37c0cb None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:46.707500 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adca707-ac43-4ecd-8b79-24311a37c0cb None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:46.707500 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adca707-ac43-4ecd-8b79-24311a37c0cb None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:46.708840 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 572/1147] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:19:46 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:19:46.714101 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964387d4-3b65-49ac-b000-eee9e764146c None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:46.714576 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964387d4-3b65-49ac-b000-eee9e764146c None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:46.714687 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964387d4-3b65-49ac-b000-eee9e764146c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:46.732431 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-964387d4-3b65-49ac-b000-eee9e764146c None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:19:46.733628 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-964387d4-3b65-49ac-b000-eee9e764146c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:19:46.800336 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 576/1148] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:19:46 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:19:47.071019 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adca707-ac43-4ecd-8b79-24311a37c0cb None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:47.137758 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:47.182764 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adca707-ac43-4ecd-8b79-24311a37c0cb None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:47.215661 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1078e1d4-81d9-4146-8a4e-f370c0b98bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:47.216282 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1078e1d4-81d9-4146-8a4e-f370c0b98bac None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:47.216563 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1078e1d4-81d9-4146-8a4e-f370c0b98bac None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:47.216837 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1078e1d4-81d9-4146-8a4e-f370c0b98bac None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:47.227467 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1078e1d4-81d9-4146-8a4e-f370c0b98bac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:19:47.232221 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-1078e1d4-81d9-4146-8a4e-f370c0b98bac None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:19:47.232221 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:19:47.232221 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:19:47.232221 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:19:47.232221 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:19:47.232221 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:19:47.232221 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:19:47.232221 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:19:47.232221 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:19:47.232221 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:19:47.232221 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:19:47.232221 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:19:47.232221 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:19:47.232221 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:19:47.232221 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:19:47.232221 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:19:47.233349 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:19:47.233349 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:19:47.233349 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:19:47.233349 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:19:47.233349 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:19:47.233349 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:19:47.233349 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:19:47.233349 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:19:47.233349 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:19:47.233349 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:19:47.233349 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:19:47.233349 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:19:47.233349 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:19:47.233349 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:19:47.233349 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:19:47.233349 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:19:47.235080 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:19:47.235080 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:19:47.235080 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:19:47.235080 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:19:47.235080 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:19:47.235080 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:19:47.235080 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:19:47.235080 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:19:47.235080 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:19:47.235080 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:19:47.235080 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:19:47.235080 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:19:47.235080 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 573/1149] 10.222.0.102 () {56 vars in 1101 bytes} [Thu Sep 1 17:19:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:19:47.239960 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964387d4-3b65-49ac-b000-eee9e764146c None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:47.303644 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:47.336948 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964387d4-3b65-49ac-b000-eee9e764146c None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:47.369996 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eae7792-ac6c-4a64-875d-eaadb590674d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:47.370593 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eae7792-ac6c-4a64-875d-eaadb590674d None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:47.370873 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eae7792-ac6c-4a64-875d-eaadb590674d None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:47.371145 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eae7792-ac6c-4a64-875d-eaadb590674d None admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:47.371800 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eae7792-ac6c-4a64-875d-eaadb590674d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:19:47.372108 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eae7792-ac6c-4a64-875d-eaadb590674d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:47.373488 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eae7792-ac6c-4a64-875d-eaadb590674d None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:47.381466 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 577/1150] 10.222.0.102 () {56 vars in 1104 bytes} [Thu Sep 1 17:19:47 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:19:47.387347 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1078e1d4-81d9-4146-8a4e-f370c0b98bac None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:47.485725 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1078e1d4-81d9-4146-8a4e-f370c0b98bac None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:47.499183 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:47.519398 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e9b585-329c-4979-a00c-c65850dae2ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:47.520008 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e9b585-329c-4979-a00c-c65850dae2ad None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:47.520284 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e9b585-329c-4979-a00c-c65850dae2ad None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:47.520553 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e9b585-329c-4979-a00c-c65850dae2ad None admin] PATH_INFO: `/v3/users/07b366c5b4094404b65dac9eed5dc6fd` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:47.521678 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e9b585-329c-4979-a00c-c65850dae2ad None admin] RBAC: Authorizing `identity:get_user(user_id=07b366c5b4094404b65dac9eed5dc6fd)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:47.523104 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e9b585-329c-4979-a00c-c65850dae2ad None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:47.524668 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 574/1151] 10.222.0.102 () {56 vars in 1182 bytes} [Thu Sep 1 17:19:47 2022] GET /identity/v3/users/07b366c5b4094404b65dac9eed5dc6fd => generated 287 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:19:47.530444 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eae7792-ac6c-4a64-875d-eaadb590674d None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:47.627760 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eae7792-ac6c-4a64-875d-eaadb590674d None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:47.652667 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:47.661092 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d35b71-ec8f-4125-be98-9eb99e451b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:47.661746 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d35b71-ec8f-4125-be98-9eb99e451b65 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:47.662036 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d35b71-ec8f-4125-be98-9eb99e451b65 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:47.662310 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d35b71-ec8f-4125-be98-9eb99e451b65 None admin] PATH_INFO: `/v3/projects/e6d239c17ad14a2aba9cacdaa7fa25f3` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:47.663425 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d35b71-ec8f-4125-be98-9eb99e451b65 None admin] RBAC: Authorizing `identity:get_project(project_id=e6d239c17ad14a2aba9cacdaa7fa25f3)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:47.664846 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d35b71-ec8f-4125-be98-9eb99e451b65 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:47.666507 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 578/1152] 10.222.0.102 () {56 vars in 1191 bytes} [Thu Sep 1 17:19:47 2022] GET /identity/v3/projects/e6d239c17ad14a2aba9cacdaa7fa25f3 => generated 312 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:19:47.673090 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e9b585-329c-4979-a00c-c65850dae2ad None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:47.770610 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e9b585-329c-4979-a00c-c65850dae2ad None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:47.803767 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd347a0-da61-41f3-8b47-256b8ba74c8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:47.804410 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd347a0-da61-41f3-8b47-256b8ba74c8c None admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:47.804687 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd347a0-da61-41f3-8b47-256b8ba74c8c None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:47.804958 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd347a0-da61-41f3-8b47-256b8ba74c8c None admin] PATH_INFO: `/v3/projects/e6d239c17ad14a2aba9cacdaa7fa25f3/users/07b366c5b4094404b65dac9eed5dc6fd/roles/36577a6f46d642d4b81805ede9872490` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:47.806825 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd347a0-da61-41f3-8b47-256b8ba74c8c None admin] RBAC: Authorizing `identity:create_grant(project_id=e6d239c17ad14a2aba9cacdaa7fa25f3, user_id=07b366c5b4094404b65dac9eed5dc6fd, role_id=36577a6f46d642d4b81805ede9872490)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:47.808259 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd347a0-da61-41f3-8b47-256b8ba74c8c None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:47.821320 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 575/1153] 10.222.0.102 () {58 vars in 1444 bytes} [Thu Sep 1 17:19:47 2022] PUT /identity/v3/projects/e6d239c17ad14a2aba9cacdaa7fa25f3/users/07b366c5b4094404b65dac9eed5dc6fd/roles/36577a6f46d642d4b81805ede9872490 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:19:49.691781 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9119b5b0-edd0-4398-9875-9768e9a7d52e None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:49.692339 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9119b5b0-edd0-4398-9875-9768e9a7d52e None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:49.692339 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9119b5b0-edd0-4398-9875-9768e9a7d52e None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:49.693719 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 579/1154] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:19:49 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:19:49.698700 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3d2271-58df-4ebc-8efb-9a381ef19722 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:49.699114 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3d2271-58df-4ebc-8efb-9a381ef19722 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:49.699248 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3d2271-58df-4ebc-8efb-9a381ef19722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:49.717608 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-7a3d2271-58df-4ebc-8efb-9a381ef19722 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:19:49.719431 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-7a3d2271-58df-4ebc-8efb-9a381ef19722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:19:49.753928 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:49.786767 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 576/1155] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:19:49 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:19:50.054230 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9119b5b0-edd0-4398-9875-9768e9a7d52e None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:50.154588 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9119b5b0-edd0-4398-9875-9768e9a7d52e None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:50.186620 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91708ab-cb67-4582-98c0-aa0ad448a090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:50.187719 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91708ab-cb67-4582-98c0-aa0ad448a090 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:50.187993 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91708ab-cb67-4582-98c0-aa0ad448a090 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:50.188259 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91708ab-cb67-4582-98c0-aa0ad448a090 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:50.198845 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91708ab-cb67-4582-98c0-aa0ad448a090 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:19:50.204559 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-d91708ab-cb67-4582-98c0-aa0ad448a090 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:19:50.204559 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:19:50.204559 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:19:50.204559 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:19:50.204559 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:19:50.204559 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:19:50.204559 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:19:50.204559 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:19:50.204559 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:19:50.204559 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:19:50.204559 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:19:50.204559 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:19:50.204559 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:19:50.204559 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:19:50.204559 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:19:50.204559 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:19:50.205768 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:19:50.205768 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:19:50.205768 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:19:50.205768 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:19:50.205768 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:19:50.205768 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:19:50.205768 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:19:50.205768 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:19:50.205768 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:19:50.205768 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:19:50.205768 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:19:50.205768 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:19:50.205768 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:19:50.205768 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:19:50.205768 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:19:50.205768 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:19:50.206616 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:19:50.206616 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:19:50.206616 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:19:50.206616 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:19:50.206616 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:19:50.206616 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:19:50.206616 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:19:50.206616 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:19:50.206616 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:19:50.206616 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:19:50.206616 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:19:50.206616 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:19:50.206616 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 580/1156] 10.222.0.102 () {56 vars in 1101 bytes} [Thu Sep 1 17:19:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:19:50.211852 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3d2271-58df-4ebc-8efb-9a381ef19722 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:50.320396 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3d2271-58df-4ebc-8efb-9a381ef19722 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:50.353820 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6422aca9-adff-49f6-9554-932e6b17b0e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:50.354423 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6422aca9-adff-49f6-9554-932e6b17b0e5 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:50.354706 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6422aca9-adff-49f6-9554-932e6b17b0e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:50.354982 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6422aca9-adff-49f6-9554-932e6b17b0e5 None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:50.355643 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6422aca9-adff-49f6-9554-932e6b17b0e5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:19:50.355950 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6422aca9-adff-49f6-9554-932e6b17b0e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:50.357340 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6422aca9-adff-49f6-9554-932e6b17b0e5 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:50.365721 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 577/1157] 10.222.0.102 () {56 vars in 1104 bytes} [Thu Sep 1 17:19:50 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:19:50.371545 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91708ab-cb67-4582-98c0-aa0ad448a090 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:50.427828 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:50.471253 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91708ab-cb67-4582-98c0-aa0ad448a090 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:50.505041 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6289dd-3fc3-432c-8016-f359ce2d56e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:50.505684 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6289dd-3fc3-432c-8016-f359ce2d56e4 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:50.505963 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6289dd-3fc3-432c-8016-f359ce2d56e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:50.506241 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6289dd-3fc3-432c-8016-f359ce2d56e4 None admin] PATH_INFO: `/v3/users/07b366c5b4094404b65dac9eed5dc6fd` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:50.507360 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6289dd-3fc3-432c-8016-f359ce2d56e4 None admin] RBAC: Authorizing `identity:get_user(user_id=07b366c5b4094404b65dac9eed5dc6fd)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:50.508790 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6289dd-3fc3-432c-8016-f359ce2d56e4 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:50.510396 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 581/1158] 10.222.0.102 () {56 vars in 1182 bytes} [Thu Sep 1 17:19:50 2022] GET /identity/v3/users/07b366c5b4094404b65dac9eed5dc6fd => generated 287 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:19:50.516854 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6422aca9-adff-49f6-9554-932e6b17b0e5 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:50.552393 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:50.618767 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6422aca9-adff-49f6-9554-932e6b17b0e5 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:50.651850 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4954737b-281b-4abf-a41a-2f7131aa90ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:50.652447 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4954737b-281b-4abf-a41a-2f7131aa90ff None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:50.652728 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4954737b-281b-4abf-a41a-2f7131aa90ff None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:50.653001 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4954737b-281b-4abf-a41a-2f7131aa90ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:50.718975 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4954737b-281b-4abf-a41a-2f7131aa90ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:50.720416 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4954737b-281b-4abf-a41a-2f7131aa90ff None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:50.752394 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:50.821987 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 578/1159] 10.222.0.102 () {58 vars in 1287 bytes} [Thu Sep 1 17:19:50 2022] GET /identity/v3/auth/tokens => generated 2123 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 17:19:50.827837 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6289dd-3fc3-432c-8016-f359ce2d56e4 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:50.926417 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6289dd-3fc3-432c-8016-f359ce2d56e4 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:50.948453 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:50.959387 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba16e8bd-f117-43d6-9834-26acc0231c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:50.959998 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba16e8bd-f117-43d6-9834-26acc0231c8e None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:50.960278 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba16e8bd-f117-43d6-9834-26acc0231c8e None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:50.960549 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba16e8bd-f117-43d6-9834-26acc0231c8e None admin] PATH_INFO: `/v3/projects/e6d239c17ad14a2aba9cacdaa7fa25f3` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:50.961679 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba16e8bd-f117-43d6-9834-26acc0231c8e None admin] RBAC: Authorizing `identity:get_project(project_id=e6d239c17ad14a2aba9cacdaa7fa25f3)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:50.963083 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba16e8bd-f117-43d6-9834-26acc0231c8e None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:50.964680 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 582/1160] 10.222.0.102 () {56 vars in 1191 bytes} [Thu Sep 1 17:19:50 2022] GET /identity/v3/projects/e6d239c17ad14a2aba9cacdaa7fa25f3 => generated 312 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:19:50.970941 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4954737b-281b-4abf-a41a-2f7131aa90ff None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:51.066665 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4954737b-281b-4abf-a41a-2f7131aa90ff None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:51.098910 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3197105-8651-4a97-883c-3d6905e5d702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:51.099498 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3197105-8651-4a97-883c-3d6905e5d702 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:51.099773 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3197105-8651-4a97-883c-3d6905e5d702 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:51.100041 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3197105-8651-4a97-883c-3d6905e5d702 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:51.100748 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3197105-8651-4a97-883c-3d6905e5d702 None admin] RBAC: Adding query filter params (role.id=36577a6f46d642d4b81805ede9872490, scope.project.id=e6d239c17ad14a2aba9cacdaa7fa25f3, user.id=07b366c5b4094404b65dac9eed5dc6fd) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:19:51.101299 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3197105-8651-4a97-883c-3d6905e5d702 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:51.102922 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3197105-8651-4a97-883c-3d6905e5d702 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:51.104667 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:51.110519 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 579/1161] 10.222.0.102 () {56 vars in 1379 bytes} [Thu Sep 1 17:19:50 2022] GET /identity/v3/role_assignments?user.id=07b366c5b4094404b65dac9eed5dc6fd&scope.project.id=e6d239c17ad14a2aba9cacdaa7fa25f3&role.id=36577a6f46d642d4b81805ede9872490 => generated 610 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:19:53.011802 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7e6937-d947-41d4-a475-e97bda444441 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:53.012323 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7e6937-d947-41d4-a475-e97bda444441 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:53.012323 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7e6937-d947-41d4-a475-e97bda444441 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:53.013713 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 583/1162] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:19:53 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:19:53.018876 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591f34ef-90e3-43cf-8763-44cd0a5c17c0 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:53.019328 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591f34ef-90e3-43cf-8763-44cd0a5c17c0 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:53.019422 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591f34ef-90e3-43cf-8763-44cd0a5c17c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:53.037772 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-591f34ef-90e3-43cf-8763-44cd0a5c17c0 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:19:53.038992 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-591f34ef-90e3-43cf-8763-44cd0a5c17c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:19:53.103765 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 580/1163] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:19:53 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:19:53.367369 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7e6937-d947-41d4-a475-e97bda444441 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:53.466058 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7e6937-d947-41d4-a475-e97bda444441 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:53.499302 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b0183f-ce62-42de-880e-376374ae1980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:53.499897 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b0183f-ce62-42de-880e-376374ae1980 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:53.500178 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b0183f-ce62-42de-880e-376374ae1980 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:53.500451 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b0183f-ce62-42de-880e-376374ae1980 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:53.555989 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:53.565580 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b0183f-ce62-42de-880e-376374ae1980 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:53.567024 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b0183f-ce62-42de-880e-376374ae1980 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:53.661971 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 584/1164] 10.222.0.102 () {58 vars in 1287 bytes} [Thu Sep 1 17:19:53 2022] GET /identity/v3/auth/tokens => generated 2123 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 17:19:53.667830 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591f34ef-90e3-43cf-8763-44cd0a5c17c0 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:53.764281 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591f34ef-90e3-43cf-8763-44cd0a5c17c0 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:53.789943 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:53.797867 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e9570d-dcdc-4178-94ec-6cbc7829bd86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:53.798488 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e9570d-dcdc-4178-94ec-6cbc7829bd86 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:53.798771 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e9570d-dcdc-4178-94ec-6cbc7829bd86 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:53.799046 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e9570d-dcdc-4178-94ec-6cbc7829bd86 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:53.805627 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e9570d-dcdc-4178-94ec-6cbc7829bd86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:53.807053 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e9570d-dcdc-4178-94ec-6cbc7829bd86 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:53.818868 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-67e9570d-dcdc-4178-94ec-6cbc7829bd86 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 01 17:19:53.818868 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:19:53.818868 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 01 17:19:53.818868 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 01 17:19:53.818868 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:19:53.818868 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:19:53.818868 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:19:53.818868 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:19:53.818868 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 01 17:19:53.818868 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:19:53.818868 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 17:19:53.818868 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:19:53.818868 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:19:53.818868 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:19:53.818868 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:19:53.818868 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:19:53.818868 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:19:53.819976 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:19:53.819976 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:19:53.819976 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:19:53.819976 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:19:53.819976 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:19:53.819976 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:19:53.819976 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:19:53.819976 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:19:53.819976 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:19:53.819976 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:19:53.819976 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:19:53.819976 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:19:53.819976 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:19:53.819976 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:19:53.819976 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:19:53.819976 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:19:53.820916 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:19:53.820916 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:19:53.820916 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:19:53.820916 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:19:53.820916 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:19:53.820916 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:19:53.820916 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:19:53.820916 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:19:53.820916 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:19:53.820916 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:19:53.820916 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 01 17:19:53.820916 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:19:53.820916 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 01 17:19:53.820916 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:19:53.820916 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 581/1165] 10.222.0.102 () {56 vars in 1122 bytes} [Thu Sep 1 17:19:53 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:19:53.825920 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b0183f-ce62-42de-880e-376374ae1980 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:53.914027 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:53.922659 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b0183f-ce62-42de-880e-376374ae1980 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:53.956504 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3617675b-1b58-4429-8e9c-7ab0f4943196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:53.957098 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3617675b-1b58-4429-8e9c-7ab0f4943196 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:53.957405 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3617675b-1b58-4429-8e9c-7ab0f4943196 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:53.957678 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3617675b-1b58-4429-8e9c-7ab0f4943196 None admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:53.958302 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3617675b-1b58-4429-8e9c-7ab0f4943196 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:19:53.958604 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3617675b-1b58-4429-8e9c-7ab0f4943196 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:53.959953 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3617675b-1b58-4429-8e9c-7ab0f4943196 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:53.966836 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 585/1166] 10.222.0.102 () {56 vars in 1120 bytes} [Thu Sep 1 17:19:53 2022] GET /identity/v3/domains?name=swift_test => generated 126 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:19:55.842928 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6f5d05-fe40-4426-a8f7-283e4f53c145 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:55.843449 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6f5d05-fe40-4426-a8f7-283e4f53c145 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:55.843449 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6f5d05-fe40-4426-a8f7-283e4f53c145 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:55.844816 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 582/1167] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:19:55 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:19:55.851603 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ba7521-5621-466d-9b48-3fdc26c84caa None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:55.852084 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ba7521-5621-466d-9b48-3fdc26c84caa None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:55.852194 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ba7521-5621-466d-9b48-3fdc26c84caa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:55.870563 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-f3ba7521-5621-466d-9b48-3fdc26c84caa None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:19:55.871754 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-f3ba7521-5621-466d-9b48-3fdc26c84caa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:19:55.939378 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 586/1168] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:19:55 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:19:56.205577 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6f5d05-fe40-4426-a8f7-283e4f53c145 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:56.313095 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6f5d05-fe40-4426-a8f7-283e4f53c145 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:56.358920 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88727ec8-504c-4ec9-869e-f5b31e7beb92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:56.359532 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88727ec8-504c-4ec9-869e-f5b31e7beb92 None admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:56.359810 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88727ec8-504c-4ec9-869e-f5b31e7beb92 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:56.360078 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88727ec8-504c-4ec9-869e-f5b31e7beb92 None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:56.360757 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88727ec8-504c-4ec9-869e-f5b31e7beb92 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:56.362171 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88727ec8-504c-4ec9-869e-f5b31e7beb92 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:56.379272 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 583/1169] 10.222.0.102 () {60 vars in 1143 bytes} [Thu Sep 1 17:19:56 2022] POST /identity/v3/domains => generated 268 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:19:58.320395 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2432da21-17e5-4cae-bc12-250dbced0aba None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:58.320955 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2432da21-17e5-4cae-bc12-250dbced0aba None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:58.320955 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2432da21-17e5-4cae-bc12-250dbced0aba None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:58.323504 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 587/1170] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:19:58 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:19:58.331537 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017197fa-ea23-451f-8025-e84557fb2fcd None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:58.331990 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017197fa-ea23-451f-8025-e84557fb2fcd None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:58.332129 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017197fa-ea23-451f-8025-e84557fb2fcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:58.364448 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-017197fa-ea23-451f-8025-e84557fb2fcd None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:19:58.366524 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-017197fa-ea23-451f-8025-e84557fb2fcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:19:58.436537 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:58.471715 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 584/1171] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:19:58 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 17:19:58.741653 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2432da21-17e5-4cae-bc12-250dbced0aba None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:58.831063 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:19:58.849958 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2432da21-17e5-4cae-bc12-250dbced0aba None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:58.885457 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df1d690-0fcb-4edf-a335-a30812a9907d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:58.886090 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df1d690-0fcb-4edf-a335-a30812a9907d None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:58.886373 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df1d690-0fcb-4edf-a335-a30812a9907d None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:58.886646 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df1d690-0fcb-4edf-a335-a30812a9907d None admin] PATH_INFO: `/v3/domains/4fd09147f10547a0b6ee9ef20385baaa` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:58.898541 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df1d690-0fcb-4edf-a335-a30812a9907d None admin] RBAC: Authorizing `identity:get_domain(domain_id=4fd09147f10547a0b6ee9ef20385baaa)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:58.900021 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df1d690-0fcb-4edf-a335-a30812a9907d None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:58.901826 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 588/1172] 10.222.0.102 () {56 vars in 1188 bytes} [Thu Sep 1 17:19:58 2022] GET /identity/v3/domains/4fd09147f10547a0b6ee9ef20385baaa => generated 268 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:19:58.907887 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017197fa-ea23-451f-8025-e84557fb2fcd None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:19:59.008494 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017197fa-ea23-451f-8025-e84557fb2fcd None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:19:59.042257 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0949d5-75f4-4b4f-8f08-4980dc61717f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:19:59.042871 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0949d5-75f4-4b4f-8f08-4980dc61717f None admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:19:59.043159 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0949d5-75f4-4b4f-8f08-4980dc61717f None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:19:59.043429 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0949d5-75f4-4b4f-8f08-4980dc61717f None admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:19:59.044095 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0949d5-75f4-4b4f-8f08-4980dc61717f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:19:59.045641 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0949d5-75f4-4b4f-8f08-4980dc61717f None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:19:59.073479 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 585/1173] 10.222.0.102 () {60 vars in 1146 bytes} [Thu Sep 1 17:19:58 2022] POST /identity/v3/projects => generated 362 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:20:01.014962 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ac84b4-a582-4eaf-85c9-80a7bd5b185a None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:20:01.015488 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ac84b4-a582-4eaf-85c9-80a7bd5b185a None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:20:01.015488 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ac84b4-a582-4eaf-85c9-80a7bd5b185a None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:20:01.016833 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 589/1174] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:20:01 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:20:01.022166 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c71464-3dd6-4f8f-bcc2-eb29a4fcf121 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:20:01.022732 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c71464-3dd6-4f8f-bcc2-eb29a4fcf121 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:20:01.022732 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c71464-3dd6-4f8f-bcc2-eb29a4fcf121 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:20:01.041098 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-e8c71464-3dd6-4f8f-bcc2-eb29a4fcf121 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:20:01.042315 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-e8c71464-3dd6-4f8f-bcc2-eb29a4fcf121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:20:01.065620 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:20:01.112267 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 586/1175] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:20:01 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:20:01.379692 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ac84b4-a582-4eaf-85c9-80a7bd5b185a None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:20:01.485592 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ac84b4-a582-4eaf-85c9-80a7bd5b185a None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:20:01.518493 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6794461-da55-46eb-854f-0265cf349e3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:20:01.519187 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6794461-da55-46eb-854f-0265cf349e3d None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:20:01.519468 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6794461-da55-46eb-854f-0265cf349e3d None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:20:01.519738 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6794461-da55-46eb-854f-0265cf349e3d None admin] PATH_INFO: `/v3/domains/4fd09147f10547a0b6ee9ef20385baaa` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:20:01.520835 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6794461-da55-46eb-854f-0265cf349e3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=4fd09147f10547a0b6ee9ef20385baaa)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:20:01.522268 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6794461-da55-46eb-854f-0265cf349e3d None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:20:01.523818 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 590/1176] 10.222.0.102 () {56 vars in 1188 bytes} [Thu Sep 1 17:20:01 2022] GET /identity/v3/domains/4fd09147f10547a0b6ee9ef20385baaa => generated 268 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:20:01.529656 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c71464-3dd6-4f8f-bcc2-eb29a4fcf121 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:20:01.629894 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c71464-3dd6-4f8f-bcc2-eb29a4fcf121 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:20:01.664648 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5ae619-cd81-411c-9a31-cb261bc658fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:20:01.665269 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5ae619-cd81-411c-9a31-cb261bc658fa None admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:20:01.665636 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5ae619-cd81-411c-9a31-cb261bc658fa None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:20:01.665912 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5ae619-cd81-411c-9a31-cb261bc658fa None admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:20:01.666693 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5ae619-cd81-411c-9a31-cb261bc658fa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:20:01.668424 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5ae619-cd81-411c-9a31-cb261bc658fa None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:20:01.693202 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 587/1177] 10.222.0.102 () {60 vars in 1137 bytes} [Thu Sep 1 17:20:01 2022] POST /identity/v3/users => generated 312 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:20:03.640944 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfba3114-2f5c-4196-9dd3-5373f1977241 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:20:03.641523 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfba3114-2f5c-4196-9dd3-5373f1977241 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:20:03.641523 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfba3114-2f5c-4196-9dd3-5373f1977241 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:20:03.642869 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 591/1178] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:20:03 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:20:03.647928 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87b40eb-a290-493e-9fd9-441448b2f090 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:20:03.648321 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87b40eb-a290-493e-9fd9-441448b2f090 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:20:03.648483 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87b40eb-a290-493e-9fd9-441448b2f090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:20:03.666284 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-f87b40eb-a290-493e-9fd9-441448b2f090 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:20:03.667468 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-f87b40eb-a290-493e-9fd9-441448b2f090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:20:03.677985 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:20:03.734815 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 588/1179] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:20:03 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:20:03.999865 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfba3114-2f5c-4196-9dd3-5373f1977241 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:20:04.015429 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:20:04.105174 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfba3114-2f5c-4196-9dd3-5373f1977241 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:20:04.138905 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516af6fb-986b-47c4-a8cb-757bceda0a23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:20:04.139530 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516af6fb-986b-47c4-a8cb-757bceda0a23 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:20:04.139811 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516af6fb-986b-47c4-a8cb-757bceda0a23 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:20:04.140084 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516af6fb-986b-47c4-a8cb-757bceda0a23 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:20:04.150344 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516af6fb-986b-47c4-a8cb-757bceda0a23 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:20:04.155609 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-516af6fb-986b-47c4-a8cb-757bceda0a23 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:20:04.155609 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:20:04.155609 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:20:04.155609 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:20:04.155609 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:20:04.155609 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:20:04.155609 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:20:04.155609 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:20:04.155609 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:20:04.155609 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:20:04.155609 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:20:04.155609 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:20:04.155609 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:20:04.155609 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:20:04.155609 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:20:04.155609 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:20:04.156804 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:20:04.156804 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:20:04.156804 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:20:04.156804 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:20:04.156804 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:20:04.156804 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:20:04.156804 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:20:04.156804 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:20:04.156804 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:20:04.156804 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:20:04.156804 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:20:04.156804 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:20:04.156804 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:20:04.156804 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:20:04.156804 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:20:04.156804 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:20:04.158132 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:20:04.158132 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:20:04.158132 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:20:04.158132 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:20:04.158132 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:20:04.158132 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:20:04.158132 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:20:04.158132 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:20:04.158132 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:20:04.158132 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:20:04.158132 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:20:04.158132 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:20:04.158132 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 592/1180] 10.222.0.102 () {56 vars in 1101 bytes} [Thu Sep 1 17:20:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:20:04.162804 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87b40eb-a290-493e-9fd9-441448b2f090 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:20:04.260405 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87b40eb-a290-493e-9fd9-441448b2f090 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:20:04.293044 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e767d48a-9d2c-4957-a30e-778a4d03ad9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:20:04.293672 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e767d48a-9d2c-4957-a30e-778a4d03ad9b None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:20:04.293951 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e767d48a-9d2c-4957-a30e-778a4d03ad9b None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:20:04.294223 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e767d48a-9d2c-4957-a30e-778a4d03ad9b None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:20:04.294881 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e767d48a-9d2c-4957-a30e-778a4d03ad9b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:20:04.295780 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e767d48a-9d2c-4957-a30e-778a4d03ad9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:20:04.297152 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e767d48a-9d2c-4957-a30e-778a4d03ad9b None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:20:04.304930 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 589/1181] 10.222.0.102 () {56 vars in 1104 bytes} [Thu Sep 1 17:20:04 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:20:04.310727 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516af6fb-986b-47c4-a8cb-757bceda0a23 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:20:04.383167 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:20:04.409787 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516af6fb-986b-47c4-a8cb-757bceda0a23 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:20:04.448098 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94aaf29b-540d-4278-ba3f-e905ae2bfe8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:20:04.448723 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aaf29b-540d-4278-ba3f-e905ae2bfe8c None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:20:04.449002 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aaf29b-540d-4278-ba3f-e905ae2bfe8c None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:20:04.449274 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aaf29b-540d-4278-ba3f-e905ae2bfe8c None admin] PATH_INFO: `/v3/users/9df871d6f3b449f1810d7fc3595eff29` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:20:04.462225 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94aaf29b-540d-4278-ba3f-e905ae2bfe8c None admin] RBAC: Authorizing `identity:get_user(user_id=9df871d6f3b449f1810d7fc3595eff29)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:20:04.463646 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94aaf29b-540d-4278-ba3f-e905ae2bfe8c None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:20:04.465242 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 593/1182] 10.222.0.102 () {56 vars in 1182 bytes} [Thu Sep 1 17:20:04 2022] GET /identity/v3/users/9df871d6f3b449f1810d7fc3595eff29 => generated 312 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:20:04.471633 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e767d48a-9d2c-4957-a30e-778a4d03ad9b None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:20:04.483608 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:20:04.572595 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e767d48a-9d2c-4957-a30e-778a4d03ad9b None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:20:04.607571 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77d3f70-a453-4804-9c37-50ea25653ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:20:04.608178 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77d3f70-a453-4804-9c37-50ea25653ce4 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:20:04.608457 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77d3f70-a453-4804-9c37-50ea25653ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:20:04.608727 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77d3f70-a453-4804-9c37-50ea25653ce4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:20:04.674046 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77d3f70-a453-4804-9c37-50ea25653ce4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:20:04.675498 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77d3f70-a453-4804-9c37-50ea25653ce4 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:20:04.685154 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:20:04.772767 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 590/1183] 10.222.0.102 () {58 vars in 1287 bytes} [Thu Sep 1 17:20:04 2022] GET /identity/v3/auth/tokens => generated 2123 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 17:20:04.778650 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94aaf29b-540d-4278-ba3f-e905ae2bfe8c None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:20:04.875182 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94aaf29b-540d-4278-ba3f-e905ae2bfe8c None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:20:04.903745 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:20:04.909068 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2163797-bc01-46a9-9e93-f658b5298ebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:20:04.909727 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2163797-bc01-46a9-9e93-f658b5298ebc None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:20:04.910006 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2163797-bc01-46a9-9e93-f658b5298ebc None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:20:04.910277 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2163797-bc01-46a9-9e93-f658b5298ebc None admin] PATH_INFO: `/v3/projects/9f40b3f62fc74895976d3b44de4b5131` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:20:04.911377 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2163797-bc01-46a9-9e93-f658b5298ebc None admin] RBAC: Authorizing `identity:get_project(project_id=9f40b3f62fc74895976d3b44de4b5131)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:20:04.912792 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2163797-bc01-46a9-9e93-f658b5298ebc None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:20:04.914430 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 594/1184] 10.222.0.102 () {56 vars in 1191 bytes} [Thu Sep 1 17:20:04 2022] GET /identity/v3/projects/9f40b3f62fc74895976d3b44de4b5131 => generated 362 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:20:04.920702 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77d3f70-a453-4804-9c37-50ea25653ce4 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:20:05.019085 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77d3f70-a453-4804-9c37-50ea25653ce4 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:20:05.028700 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:20:05.053634 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fd6ab1-8bea-4f2b-aa1b-2441c9c7b69b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:20:05.054242 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fd6ab1-8bea-4f2b-aa1b-2441c9c7b69b None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:20:05.054516 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fd6ab1-8bea-4f2b-aa1b-2441c9c7b69b None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:20:05.054787 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fd6ab1-8bea-4f2b-aa1b-2441c9c7b69b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:20:05.055497 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fd6ab1-8bea-4f2b-aa1b-2441c9c7b69b None admin] RBAC: Adding query filter params (role.id=36577a6f46d642d4b81805ede9872490, scope.project.id=9f40b3f62fc74895976d3b44de4b5131, user.id=9df871d6f3b449f1810d7fc3595eff29) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:20:05.055803 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fd6ab1-8bea-4f2b-aa1b-2441c9c7b69b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:20:05.057218 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fd6ab1-8bea-4f2b-aa1b-2441c9c7b69b None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:20:05.063035 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 591/1185] 10.222.0.102 () {56 vars in 1379 bytes} [Thu Sep 1 17:20:04 2022] GET /identity/v3/role_assignments?user.id=9df871d6f3b449f1810d7fc3595eff29&scope.project.id=9f40b3f62fc74895976d3b44de4b5131&role.id=36577a6f46d642d4b81805ede9872490 => generated 260 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:20:06.908944 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4667c40a-9863-4dc5-91ae-e9a9310e3502 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:20:06.909491 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4667c40a-9863-4dc5-91ae-e9a9310e3502 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:20:06.909619 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4667c40a-9863-4dc5-91ae-e9a9310e3502 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:20:06.910857 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 595/1186] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:20:06 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:20:06.915886 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b3f70f-7aa3-4d99-a842-f6fb2afdbda1 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:20:06.916311 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b3f70f-7aa3-4d99-a842-f6fb2afdbda1 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:20:06.916438 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b3f70f-7aa3-4d99-a842-f6fb2afdbda1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:20:06.934444 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-34b3f70f-7aa3-4d99-a842-f6fb2afdbda1 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:20:06.935752 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-34b3f70f-7aa3-4d99-a842-f6fb2afdbda1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:20:07.001282 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 592/1187] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:20:06 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:20:07.273253 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4667c40a-9863-4dc5-91ae-e9a9310e3502 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:20:07.389122 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4667c40a-9863-4dc5-91ae-e9a9310e3502 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:20:07.424860 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5017162-e989-40da-8ceb-9b6ecf46f176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:20:07.425516 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5017162-e989-40da-8ceb-9b6ecf46f176 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:20:07.425795 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5017162-e989-40da-8ceb-9b6ecf46f176 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:20:07.426066 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5017162-e989-40da-8ceb-9b6ecf46f176 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:20:07.437059 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5017162-e989-40da-8ceb-9b6ecf46f176 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:20:07.441889 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-a5017162-e989-40da-8ceb-9b6ecf46f176 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:20:07.441889 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:20:07.441889 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:20:07.441889 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:20:07.441889 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:20:07.441889 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:20:07.441889 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:20:07.441889 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:20:07.441889 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:20:07.441889 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:20:07.441889 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:20:07.441889 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:20:07.441889 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:20:07.441889 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:20:07.441889 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:20:07.441889 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:20:07.443748 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:20:07.443748 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:20:07.443748 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:20:07.443748 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:20:07.443748 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:20:07.443748 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:20:07.443748 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:20:07.443748 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:20:07.443748 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:20:07.443748 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:20:07.443748 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:20:07.443748 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:20:07.443748 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:20:07.443748 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:20:07.443748 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:20:07.443748 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:20:07.445855 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:20:07.445855 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:20:07.445855 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:20:07.445855 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:20:07.445855 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:20:07.445855 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:20:07.445855 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:20:07.445855 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:20:07.445855 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:20:07.445855 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:20:07.445855 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:20:07.445855 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:20:07.445855 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 596/1188] 10.222.0.102 () {56 vars in 1101 bytes} [Thu Sep 1 17:20:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:20:07.450894 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b3f70f-7aa3-4d99-a842-f6fb2afdbda1 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:20:07.561955 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b3f70f-7aa3-4d99-a842-f6fb2afdbda1 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:20:07.563520 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:20:07.606861 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a375ac-06e1-45a7-81d2-ed044d086251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:20:07.607482 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a375ac-06e1-45a7-81d2-ed044d086251 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:20:07.607765 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a375ac-06e1-45a7-81d2-ed044d086251 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:20:07.608037 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a375ac-06e1-45a7-81d2-ed044d086251 None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:20:07.608815 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a375ac-06e1-45a7-81d2-ed044d086251 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:20:07.609134 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a375ac-06e1-45a7-81d2-ed044d086251 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:20:07.610558 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a375ac-06e1-45a7-81d2-ed044d086251 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:20:07.619418 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 593/1189] 10.222.0.102 () {56 vars in 1104 bytes} [Thu Sep 1 17:20:07 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:20:07.625415 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5017162-e989-40da-8ceb-9b6ecf46f176 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:20:07.701072 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:20:07.753791 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5017162-e989-40da-8ceb-9b6ecf46f176 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:20:07.790448 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd8da1a-6ae1-4e53-b961-8fa68567eb8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:20:07.791072 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd8da1a-6ae1-4e53-b961-8fa68567eb8b None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:20:07.791348 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd8da1a-6ae1-4e53-b961-8fa68567eb8b None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:20:07.791619 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd8da1a-6ae1-4e53-b961-8fa68567eb8b None admin] PATH_INFO: `/v3/users/9df871d6f3b449f1810d7fc3595eff29` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:20:07.792750 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd8da1a-6ae1-4e53-b961-8fa68567eb8b None admin] RBAC: Authorizing `identity:get_user(user_id=9df871d6f3b449f1810d7fc3595eff29)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:20:07.794195 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd8da1a-6ae1-4e53-b961-8fa68567eb8b None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:20:07.795872 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 597/1190] 10.222.0.102 () {56 vars in 1182 bytes} [Thu Sep 1 17:20:07 2022] GET /identity/v3/users/9df871d6f3b449f1810d7fc3595eff29 => generated 312 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:20:07.801863 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a375ac-06e1-45a7-81d2-ed044d086251 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:20:07.906274 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a375ac-06e1-45a7-81d2-ed044d086251 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:20:07.940742 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afde760e-e232-4681-bbe6-2536f4c3fdc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:20:07.941421 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afde760e-e232-4681-bbe6-2536f4c3fdc6 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:20:07.941678 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afde760e-e232-4681-bbe6-2536f4c3fdc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:20:07.941953 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afde760e-e232-4681-bbe6-2536f4c3fdc6 None admin] PATH_INFO: `/v3/projects/9f40b3f62fc74895976d3b44de4b5131` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:20:07.943077 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afde760e-e232-4681-bbe6-2536f4c3fdc6 None admin] RBAC: Authorizing `identity:get_project(project_id=9f40b3f62fc74895976d3b44de4b5131)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:20:07.944510 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afde760e-e232-4681-bbe6-2536f4c3fdc6 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:20:07.946196 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 594/1191] 10.222.0.102 () {56 vars in 1191 bytes} [Thu Sep 1 17:20:07 2022] GET /identity/v3/projects/9f40b3f62fc74895976d3b44de4b5131 => generated 362 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:20:07.952731 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd8da1a-6ae1-4e53-b961-8fa68567eb8b None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:20:08.051807 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd8da1a-6ae1-4e53-b961-8fa68567eb8b None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:20:08.078745 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:20:08.087563 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feeacc80-3fa2-46b5-86b0-a7883d148c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:20:08.088218 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeacc80-3fa2-46b5-86b0-a7883d148c7a None admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:20:08.088495 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeacc80-3fa2-46b5-86b0-a7883d148c7a None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:20:08.088765 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeacc80-3fa2-46b5-86b0-a7883d148c7a None admin] PATH_INFO: `/v3/projects/9f40b3f62fc74895976d3b44de4b5131/users/9df871d6f3b449f1810d7fc3595eff29/roles/36577a6f46d642d4b81805ede9872490` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:20:08.090803 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feeacc80-3fa2-46b5-86b0-a7883d148c7a None admin] RBAC: Authorizing `identity:create_grant(project_id=9f40b3f62fc74895976d3b44de4b5131, user_id=9df871d6f3b449f1810d7fc3595eff29, role_id=36577a6f46d642d4b81805ede9872490)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:20:08.092258 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feeacc80-3fa2-46b5-86b0-a7883d148c7a None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:20:08.106319 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 598/1192] 10.222.0.102 () {58 vars in 1444 bytes} [Thu Sep 1 17:20:07 2022] PUT /identity/v3/projects/9f40b3f62fc74895976d3b44de4b5131/users/9df871d6f3b449f1810d7fc3595eff29/roles/36577a6f46d642d4b81805ede9872490 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:20:10.005080 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8445d30d-9baa-4b0c-9675-809af23f4025 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:20:10.005681 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8445d30d-9baa-4b0c-9675-809af23f4025 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:20:10.005681 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8445d30d-9baa-4b0c-9675-809af23f4025 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:20:10.007021 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 595/1193] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:20:10 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:20:10.012336 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7591ef4-7bf7-4e55-a811-d457ba7fc73b None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:20:10.012773 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7591ef4-7bf7-4e55-a811-d457ba7fc73b None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:20:10.012892 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7591ef4-7bf7-4e55-a811-d457ba7fc73b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:20:10.036412 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-b7591ef4-7bf7-4e55-a811-d457ba7fc73b None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:20:10.037634 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-b7591ef4-7bf7-4e55-a811-d457ba7fc73b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:20:10.106858 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 599/1194] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:20:10 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:20:10.374333 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8445d30d-9baa-4b0c-9675-809af23f4025 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:20:10.406953 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:20:10.478625 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8445d30d-9baa-4b0c-9675-809af23f4025 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:20:10.512206 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c760c2e1-d909-4b61-a7bb-ff1199f272a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:20:10.512835 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c760c2e1-d909-4b61-a7bb-ff1199f272a6 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:20:10.513118 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c760c2e1-d909-4b61-a7bb-ff1199f272a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:20:10.513433 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c760c2e1-d909-4b61-a7bb-ff1199f272a6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:20:10.525206 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c760c2e1-d909-4b61-a7bb-ff1199f272a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:20:10.530598 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-c760c2e1-d909-4b61-a7bb-ff1199f272a6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:20:10.530598 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:20:10.530598 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:20:10.530598 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:20:10.530598 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:20:10.530598 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:20:10.530598 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:20:10.530598 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:20:10.530598 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:20:10.530598 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:20:10.530598 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:20:10.530598 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:20:10.530598 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:20:10.530598 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:20:10.530598 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:20:10.530598 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:20:10.532535 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:20:10.532535 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:20:10.532535 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:20:10.532535 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:20:10.532535 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:20:10.532535 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:20:10.532535 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:20:10.532535 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:20:10.532535 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:20:10.532535 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:20:10.532535 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:20:10.532535 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:20:10.532535 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:20:10.532535 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:20:10.532535 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:20:10.532535 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:20:10.534110 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:20:10.534110 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:20:10.534110 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:20:10.534110 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:20:10.534110 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:20:10.534110 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:20:10.534110 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:20:10.534110 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:20:10.534110 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:20:10.534110 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:20:10.534110 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:20:10.534110 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:20:10.534110 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 596/1195] 10.222.0.102 () {56 vars in 1101 bytes} [Thu Sep 1 17:20:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:20:10.537936 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7591ef4-7bf7-4e55-a811-d457ba7fc73b None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:20:10.635918 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7591ef4-7bf7-4e55-a811-d457ba7fc73b None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:20:10.653046 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:20:10.670764 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b930ec-05cc-4f38-9651-664518dbb464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:20:10.671365 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b930ec-05cc-4f38-9651-664518dbb464 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:20:10.671646 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b930ec-05cc-4f38-9651-664518dbb464 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:20:10.671919 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b930ec-05cc-4f38-9651-664518dbb464 None admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:20:10.672584 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b930ec-05cc-4f38-9651-664518dbb464 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:20:10.672885 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b930ec-05cc-4f38-9651-664518dbb464 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:20:10.674282 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b930ec-05cc-4f38-9651-664518dbb464 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:20:10.682616 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 600/1196] 10.222.0.102 () {56 vars in 1104 bytes} [Thu Sep 1 17:20:10 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:20:10.688863 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c760c2e1-d909-4b61-a7bb-ff1199f272a6 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:20:10.783941 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:20:10.792927 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c760c2e1-d909-4b61-a7bb-ff1199f272a6 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:20:10.827723 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354366fd-8660-48e6-993c-d676f65e3e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:20:10.828351 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354366fd-8660-48e6-993c-d676f65e3e38 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:20:10.828625 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354366fd-8660-48e6-993c-d676f65e3e38 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:20:10.828892 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354366fd-8660-48e6-993c-d676f65e3e38 None admin] PATH_INFO: `/v3/users/9df871d6f3b449f1810d7fc3595eff29` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:20:10.830006 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354366fd-8660-48e6-993c-d676f65e3e38 None admin] RBAC: Authorizing `identity:get_user(user_id=9df871d6f3b449f1810d7fc3595eff29)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:20:10.831434 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354366fd-8660-48e6-993c-d676f65e3e38 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:20:10.833018 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 597/1197] 10.222.0.102 () {56 vars in 1182 bytes} [Thu Sep 1 17:20:10 2022] GET /identity/v3/users/9df871d6f3b449f1810d7fc3595eff29 => generated 312 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:20:10.839225 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b930ec-05cc-4f38-9651-664518dbb464 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:20:10.940824 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b930ec-05cc-4f38-9651-664518dbb464 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:20:10.973970 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1c3861-64c3-4f75-b991-e941e74e9f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:20:10.975543 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1c3861-64c3-4f75-b991-e941e74e9f35 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:20:10.975543 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1c3861-64c3-4f75-b991-e941e74e9f35 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:20:10.975543 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1c3861-64c3-4f75-b991-e941e74e9f35 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:20:11.024682 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:20:11.044761 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1c3861-64c3-4f75-b991-e941e74e9f35 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:20:11.046222 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1c3861-64c3-4f75-b991-e941e74e9f35 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:20:11.146039 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 601/1198] 10.222.0.102 () {58 vars in 1287 bytes} [Thu Sep 1 17:20:10 2022] GET /identity/v3/auth/tokens => generated 2123 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 01 17:20:11.152152 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354366fd-8660-48e6-993c-d676f65e3e38 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:20:11.270071 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354366fd-8660-48e6-993c-d676f65e3e38 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:20:11.306630 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233d5753-3c7c-4116-9679-b32def664352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:20:11.307267 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233d5753-3c7c-4116-9679-b32def664352 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:20:11.307549 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233d5753-3c7c-4116-9679-b32def664352 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:20:11.307824 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233d5753-3c7c-4116-9679-b32def664352 None admin] PATH_INFO: `/v3/projects/9f40b3f62fc74895976d3b44de4b5131` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:20:11.308988 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233d5753-3c7c-4116-9679-b32def664352 None admin] RBAC: Authorizing `identity:get_project(project_id=9f40b3f62fc74895976d3b44de4b5131)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:20:11.311045 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233d5753-3c7c-4116-9679-b32def664352 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:20:11.312722 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 598/1199] 10.222.0.102 () {56 vars in 1191 bytes} [Thu Sep 1 17:20:11 2022] GET /identity/v3/projects/9f40b3f62fc74895976d3b44de4b5131 => generated 362 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:20:11.319588 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1c3861-64c3-4f75-b991-e941e74e9f35 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:20:11.401297 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:20:11.421510 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1c3861-64c3-4f75-b991-e941e74e9f35 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:20:11.461260 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ff5daa-7173-490b-8f7b-aae3ac86ac41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:20:11.461904 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ff5daa-7173-490b-8f7b-aae3ac86ac41 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:20:11.462178 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ff5daa-7173-490b-8f7b-aae3ac86ac41 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:20:11.462449 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ff5daa-7173-490b-8f7b-aae3ac86ac41 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:20:11.463176 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ff5daa-7173-490b-8f7b-aae3ac86ac41 None admin] RBAC: Adding query filter params (role.id=36577a6f46d642d4b81805ede9872490, scope.project.id=9f40b3f62fc74895976d3b44de4b5131, user.id=9df871d6f3b449f1810d7fc3595eff29) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:20:11.463513 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ff5daa-7173-490b-8f7b-aae3ac86ac41 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:20:11.464932 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ff5daa-7173-490b-8f7b-aae3ac86ac41 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:20:11.471095 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 602/1200] 10.222.0.102 () {56 vars in 1379 bytes} [Thu Sep 1 17:20:11 2022] GET /identity/v3/role_assignments?user.id=9df871d6f3b449f1810d7fc3595eff29&scope.project.id=9f40b3f62fc74895976d3b44de4b5131&role.id=36577a6f46d642d4b81805ede9872490 => generated 610 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:20:21.302639 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=86 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:20:21.468347 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=28 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:21:07.613371 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a90a28-807c-4650-bc39-7f1342966248 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:21:07.613371 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a90a28-807c-4650-bc39-7f1342966248 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:21:07.613371 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a90a28-807c-4650-bc39-7f1342966248 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:21:07.614363 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 599/1201] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:21:07 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:21:07.620020 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71d308f-6f11-464a-98d9-4b7a0e89f59e None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:21:07.620895 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71d308f-6f11-464a-98d9-4b7a0e89f59e None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:21:07.621487 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71d308f-6f11-464a-98d9-4b7a0e89f59e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:21:07.641903 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-e71d308f-6f11-464a-98d9-4b7a0e89f59e None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:21:07.643095 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-e71d308f-6f11-464a-98d9-4b7a0e89f59e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:21:07.725888 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 603/1202] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:21:07 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:21:08.006699 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a90a28-807c-4650-bc39-7f1342966248 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:21:08.136702 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a90a28-807c-4650-bc39-7f1342966248 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:21:08.182932 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bab522-0059-4f06-8780-b2f354bfc760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:21:08.183580 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bab522-0059-4f06-8780-b2f354bfc760 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:21:08.183863 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bab522-0059-4f06-8780-b2f354bfc760 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:21:08.184139 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bab522-0059-4f06-8780-b2f354bfc760 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:21:08.199114 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bab522-0059-4f06-8780-b2f354bfc760 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:21:08.199550 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bab522-0059-4f06-8780-b2f354bfc760 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:21:08.219260 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-e9bab522-0059-4f06-8780-b2f354bfc760 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:21:08.219260 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:21:08.219260 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:21:08.219260 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:21:08.219260 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:21:08.219260 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:21:08.219260 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:21:08.219260 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:21:08.219260 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:21:08.219260 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:21:08.219260 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:21:08.219260 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:21:08.219260 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:21:08.219260 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:21:08.219260 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:21:08.219260 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:21:08.220691 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:21:08.220691 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:21:08.220691 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:21:08.220691 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:21:08.220691 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:21:08.220691 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:21:08.220691 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:21:08.220691 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:21:08.220691 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:21:08.220691 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:21:08.220691 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:21:08.220691 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:21:08.220691 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:21:08.220691 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:21:08.220691 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:21:08.220691 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:21:08.220691 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:21:08.221832 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:21:08.221832 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:21:08.221832 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:21:08.225527 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 600/1203] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:21:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:21:08.227187 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71d308f-6f11-464a-98d9-4b7a0e89f59e None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:21:08.358826 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71d308f-6f11-464a-98d9-4b7a0e89f59e None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:21:08.391405 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:21:08.399969 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205fa0ff-b069-436c-a184-da7dbcc8ac56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:21:08.405932 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205fa0ff-b069-436c-a184-da7dbcc8ac56 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:21:08.405932 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205fa0ff-b069-436c-a184-da7dbcc8ac56 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:21:08.405932 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205fa0ff-b069-436c-a184-da7dbcc8ac56 None admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:21:08.405932 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205fa0ff-b069-436c-a184-da7dbcc8ac56 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:21:08.405932 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205fa0ff-b069-436c-a184-da7dbcc8ac56 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:21:08.405932 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205fa0ff-b069-436c-a184-da7dbcc8ac56 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:21:08.419101 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 604/1204] 10.222.0.102 () {56 vars in 1114 bytes} [Thu Sep 1 17:21:08 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:21:08.425804 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bab522-0059-4f06-8780-b2f354bfc760 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:21:08.504010 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:21:08.565067 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bab522-0059-4f06-8780-b2f354bfc760 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:21:08.609910 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fb3d4e-cb1d-455f-bd40-ccd6c3cd5a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:21:08.610918 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fb3d4e-cb1d-455f-bd40-ccd6c3cd5a36 None admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:21:08.611461 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fb3d4e-cb1d-455f-bd40-ccd6c3cd5a36 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:21:08.611949 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fb3d4e-cb1d-455f-bd40-ccd6c3cd5a36 None admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:21:08.612920 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fb3d4e-cb1d-455f-bd40-ccd6c3cd5a36 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:21:08.614693 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fb3d4e-cb1d-455f-bd40-ccd6c3cd5a36 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:21:08.642729 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 601/1205] 10.222.0.102 () {60 vars in 1137 bytes} [Thu Sep 1 17:21:08 2022] POST /identity/v3/users => generated 252 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:21:10.749649 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4c4d5c-0d1e-4bb3-809d-8eab9f2c3450 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:21:10.750244 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4c4d5c-0d1e-4bb3-809d-8eab9f2c3450 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:21:10.750244 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4c4d5c-0d1e-4bb3-809d-8eab9f2c3450 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:21:10.751823 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 605/1206] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:21:10 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:21:10.757219 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b177686-cab3-486c-bfb8-8b16ddbf3c43 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:21:10.757739 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b177686-cab3-486c-bfb8-8b16ddbf3c43 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:21:10.757828 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b177686-cab3-486c-bfb8-8b16ddbf3c43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:21:10.778143 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-8b177686-cab3-486c-bfb8-8b16ddbf3c43 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:21:10.779522 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-8b177686-cab3-486c-bfb8-8b16ddbf3c43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:21:10.862551 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 602/1207] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:21:10 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 17:21:11.176274 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4c4d5c-0d1e-4bb3-809d-8eab9f2c3450 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:21:11.311980 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4c4d5c-0d1e-4bb3-809d-8eab9f2c3450 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:21:11.354499 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a62ffd1-9bec-4ab6-bc6b-d5900fa75b6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:21:11.355334 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a62ffd1-9bec-4ab6-bc6b-d5900fa75b6a None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:21:11.355687 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a62ffd1-9bec-4ab6-bc6b-d5900fa75b6a None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:21:11.356263 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a62ffd1-9bec-4ab6-bc6b-d5900fa75b6a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:21:11.368664 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a62ffd1-9bec-4ab6-bc6b-d5900fa75b6a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:21:11.375324 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-3a62ffd1-9bec-4ab6-bc6b-d5900fa75b6a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:21:11.375324 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:21:11.375324 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:21:11.375324 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:21:11.375324 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:21:11.375324 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:21:11.375324 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:21:11.375324 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:21:11.375324 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:21:11.375324 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:21:11.375324 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:21:11.375324 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:21:11.375324 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:21:11.375324 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:21:11.375324 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:21:11.375324 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:21:11.376632 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:21:11.376632 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:21:11.376632 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:21:11.376632 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:21:11.376632 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:21:11.376632 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:21:11.376632 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:21:11.376632 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:21:11.376632 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:21:11.376632 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:21:11.376632 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:21:11.376632 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:21:11.376632 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:21:11.376632 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:21:11.376632 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:21:11.376632 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:21:11.378019 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:21:11.378019 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:21:11.378019 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:21:11.378019 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:21:11.378019 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:21:11.378019 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:21:11.378019 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:21:11.378019 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:21:11.378019 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:21:11.378019 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:21:11.378019 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:21:11.378019 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:21:11.378019 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 606/1208] 10.222.0.102 () {56 vars in 1107 bytes} [Thu Sep 1 17:21:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:21:11.385991 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b177686-cab3-486c-bfb8-8b16ddbf3c43 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:21:11.433335 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:21:11.511626 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b177686-cab3-486c-bfb8-8b16ddbf3c43 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:21:11.550920 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d074fc0a-a776-41e1-bb8a-ab2d877cdb29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:21:11.551601 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d074fc0a-a776-41e1-bb8a-ab2d877cdb29 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:21:11.552413 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d074fc0a-a776-41e1-bb8a-ab2d877cdb29 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:21:11.553036 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d074fc0a-a776-41e1-bb8a-ab2d877cdb29 None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:21:11.553743 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d074fc0a-a776-41e1-bb8a-ab2d877cdb29 None admin] RBAC: Adding query filter params (name=service) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:21:11.554076 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d074fc0a-a776-41e1-bb8a-ab2d877cdb29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:21:11.555558 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d074fc0a-a776-41e1-bb8a-ab2d877cdb29 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:21:11.571856 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 603/1209] 10.222.0.102 () {56 vars in 1108 bytes} [Thu Sep 1 17:21:11 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:21:11.573241 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a62ffd1-9bec-4ab6-bc6b-d5900fa75b6a None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:21:11.638560 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:21:11.707089 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a62ffd1-9bec-4ab6-bc6b-d5900fa75b6a None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:21:11.754113 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51c2ede-69c7-4ba9-8bf8-3ba41c04dc78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:21:11.756084 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51c2ede-69c7-4ba9-8bf8-3ba41c04dc78 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:21:11.756084 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51c2ede-69c7-4ba9-8bf8-3ba41c04dc78 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:21:11.756084 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51c2ede-69c7-4ba9-8bf8-3ba41c04dc78 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:21:11.771571 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51c2ede-69c7-4ba9-8bf8-3ba41c04dc78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:21:11.773570 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51c2ede-69c7-4ba9-8bf8-3ba41c04dc78 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:21:11.792923 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-c51c2ede-69c7-4ba9-8bf8-3ba41c04dc78 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:21:11.792923 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:21:11.792923 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:21:11.792923 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:21:11.792923 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:21:11.792923 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:21:11.792923 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:21:11.792923 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:21:11.792923 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:21:11.792923 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:21:11.792923 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:21:11.792923 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:21:11.792923 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:21:11.792923 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:21:11.792923 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:21:11.792923 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:21:11.794979 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:21:11.794979 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:21:11.794979 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:21:11.794979 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:21:11.794979 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:21:11.794979 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:21:11.794979 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:21:11.794979 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:21:11.794979 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:21:11.794979 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:21:11.794979 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:21:11.794979 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:21:11.794979 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:21:11.794979 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:21:11.794979 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:21:11.794979 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:21:11.794979 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:21:11.795905 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:21:11.795905 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:21:11.795905 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:21:11.795905 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 607/1210] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:21:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:21:11.802068 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d074fc0a-a776-41e1-bb8a-ab2d877cdb29 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:21:11.919772 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:21:11.923908 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d074fc0a-a776-41e1-bb8a-ab2d877cdb29 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:21:11.971404 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50a34e7-ba3e-49e5-9cb3-fcd8af9ddd4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:21:11.971404 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50a34e7-ba3e-49e5-9cb3-fcd8af9ddd4f None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:21:11.971997 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50a34e7-ba3e-49e5-9cb3-fcd8af9ddd4f None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:21:11.972644 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50a34e7-ba3e-49e5-9cb3-fcd8af9ddd4f None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:21:11.974038 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50a34e7-ba3e-49e5-9cb3-fcd8af9ddd4f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:21:11.974479 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50a34e7-ba3e-49e5-9cb3-fcd8af9ddd4f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:21:11.976931 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50a34e7-ba3e-49e5-9cb3-fcd8af9ddd4f None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:21:11.996279 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 604/1211] 10.222.0.102 () {56 vars in 1114 bytes} [Thu Sep 1 17:21:11 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:21:12.001329 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51c2ede-69c7-4ba9-8bf8-3ba41c04dc78 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:21:12.128917 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51c2ede-69c7-4ba9-8bf8-3ba41c04dc78 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:21:12.132066 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:21:12.174908 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a7c746-f5ed-4461-b7ae-3182ef68c668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:21:12.175550 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a7c746-f5ed-4461-b7ae-3182ef68c668 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:21:12.175832 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a7c746-f5ed-4461-b7ae-3182ef68c668 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:21:12.176097 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a7c746-f5ed-4461-b7ae-3182ef68c668 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:21:12.184791 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a7c746-f5ed-4461-b7ae-3182ef68c668 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:21:12.186355 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a7c746-f5ed-4461-b7ae-3182ef68c668 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:21:12.205394 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-07a7c746-f5ed-4461-b7ae-3182ef68c668 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 01 17:21:12.205394 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:21:12.205394 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:21:12.205394 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:21:12.205394 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:21:12.205394 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:21:12.205394 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:21:12.205394 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:21:12.205394 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:21:12.205394 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:21:12.205394 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:21:12.205394 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:21:12.205394 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:21:12.205394 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:21:12.205394 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:21:12.205394 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:21:12.211521 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:21:12.211521 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:21:12.211521 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:21:12.211521 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:21:12.211521 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:21:12.211521 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:21:12.211521 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:21:12.211521 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:21:12.211521 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:21:12.211521 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:21:12.211521 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:21:12.211521 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:21:12.211521 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:21:12.211521 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:21:12.211521 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:21:12.211521 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:21:12.211521 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:21:12.211521 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:21:12.212701 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:21:12.212701 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:21:12.212701 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:21:12.212701 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:21:12.212701 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:21:12.212701 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:21:12.212701 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:21:12.212701 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:21:12.212701 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 01 17:21:12.212701 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:21:12.212701 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 608/1212] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:21:11 2022] GET /identity/v3/users/placement => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:21:12.213443 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50a34e7-ba3e-49e5-9cb3-fcd8af9ddd4f None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:21:12.344685 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50a34e7-ba3e-49e5-9cb3-fcd8af9ddd4f None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:21:12.384245 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d67e5b-4605-4131-9657-ca238be99a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:21:12.384863 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d67e5b-4605-4131-9657-ca238be99a32 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:21:12.385149 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d67e5b-4605-4131-9657-ca238be99a32 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:21:12.385456 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d67e5b-4605-4131-9657-ca238be99a32 None admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:21:12.386193 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d67e5b-4605-4131-9657-ca238be99a32 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:21:12.386507 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d67e5b-4605-4131-9657-ca238be99a32 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:21:12.387943 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d67e5b-4605-4131-9657-ca238be99a32 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:21:12.408191 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 605/1213] 10.222.0.102 () {56 vars in 1148 bytes} [Thu Sep 1 17:21:12 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 380 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:21:12.414697 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a7c746-f5ed-4461-b7ae-3182ef68c668 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:21:12.538866 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a7c746-f5ed-4461-b7ae-3182ef68c668 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:21:12.582956 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9446ba78-5b19-42a9-9a3f-345bb3d01617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:21:12.583743 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9446ba78-5b19-42a9-9a3f-345bb3d01617 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:21:12.584165 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9446ba78-5b19-42a9-9a3f-345bb3d01617 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:21:12.584605 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9446ba78-5b19-42a9-9a3f-345bb3d01617 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:21:12.625836 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:21:12.669911 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9446ba78-5b19-42a9-9a3f-345bb3d01617 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:21:12.671398 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9446ba78-5b19-42a9-9a3f-345bb3d01617 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:21:12.818838 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 609/1214] 10.222.0.102 () {58 vars in 1287 bytes} [Thu Sep 1 17:21:12 2022] GET /identity/v3/auth/tokens => generated 2123 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 17:21:12.819798 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d67e5b-4605-4131-9657-ca238be99a32 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:21:12.840433 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:21:12.942191 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d67e5b-4605-4131-9657-ca238be99a32 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:21:12.986897 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a0bf57-efcb-45e2-b1c5-394bec1a6403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:21:12.987567 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a0bf57-efcb-45e2-b1c5-394bec1a6403 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:21:12.987850 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a0bf57-efcb-45e2-b1c5-394bec1a6403 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:21:12.988137 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a0bf57-efcb-45e2-b1c5-394bec1a6403 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:21:13.001020 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a0bf57-efcb-45e2-b1c5-394bec1a6403 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:21:13.002657 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a0bf57-efcb-45e2-b1c5-394bec1a6403 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:21:13.023857 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-40a0bf57-efcb-45e2-b1c5-394bec1a6403 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:21:13.023857 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:21:13.023857 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:21:13.023857 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:21:13.023857 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:21:13.023857 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:21:13.023857 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:21:13.023857 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:21:13.023857 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:21:13.023857 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:21:13.023857 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:21:13.023857 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:21:13.023857 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:21:13.023857 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:21:13.023857 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:21:13.023857 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:21:13.024953 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:21:13.024953 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:21:13.024953 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:21:13.024953 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:21:13.024953 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:21:13.024953 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:21:13.024953 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:21:13.024953 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:21:13.024953 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:21:13.024953 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:21:13.024953 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:21:13.024953 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:21:13.024953 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:21:13.024953 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:21:13.024953 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:21:13.024953 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:21:13.024953 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:21:13.026361 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:21:13.026361 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:21:13.026361 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:21:13.026361 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 606/1215] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:21:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:21:13.031357 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9446ba78-5b19-42a9-9a3f-345bb3d01617 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:21:13.101909 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:21:13.143464 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9446ba78-5b19-42a9-9a3f-345bb3d01617 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:21:13.183134 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd782ea-f890-4953-b068-508970f6977f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:21:13.183767 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd782ea-f890-4953-b068-508970f6977f None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:21:13.183998 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd782ea-f890-4953-b068-508970f6977f None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:21:13.184319 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd782ea-f890-4953-b068-508970f6977f None admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:21:13.184962 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd782ea-f890-4953-b068-508970f6977f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:21:13.185249 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd782ea-f890-4953-b068-508970f6977f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:21:13.186779 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd782ea-f890-4953-b068-508970f6977f None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:21:13.199310 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 610/1216] 10.222.0.102 () {56 vars in 1114 bytes} [Thu Sep 1 17:21:13 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:21:13.206211 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a0bf57-efcb-45e2-b1c5-394bec1a6403 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:21:13.289096 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:21:13.325746 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a0bf57-efcb-45e2-b1c5-394bec1a6403 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:21:13.381034 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afd8c66-eb85-4d73-84a7-a7b1139ab73c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:21:13.382100 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afd8c66-eb85-4d73-84a7-a7b1139ab73c None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:21:13.382587 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afd8c66-eb85-4d73-84a7-a7b1139ab73c None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:21:13.383054 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afd8c66-eb85-4d73-84a7-a7b1139ab73c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:21:13.394772 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afd8c66-eb85-4d73-84a7-a7b1139ab73c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:21:13.396560 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afd8c66-eb85-4d73-84a7-a7b1139ab73c None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:21:13.409986 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-6afd8c66-eb85-4d73-84a7-a7b1139ab73c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:21:13.409986 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:21:13.409986 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:21:13.409986 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:21:13.409986 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:21:13.409986 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:21:13.409986 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:21:13.409986 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:21:13.409986 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:21:13.409986 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:21:13.409986 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:21:13.409986 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:21:13.409986 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:21:13.409986 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:21:13.409986 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:21:13.416101 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:21:13.416101 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:21:13.416101 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:21:13.416101 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:21:13.416101 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:21:13.416101 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:21:13.416101 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:21:13.416101 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:21:13.416101 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:21:13.416101 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:21:13.416101 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:21:13.416101 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:21:13.416101 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:21:13.416101 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:21:13.416101 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:21:13.416101 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:21:13.416101 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:21:13.418155 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:21:13.418155 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:21:13.418155 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:21:13.418155 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:21:13.418155 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:21:13.418155 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:21:13.418155 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:21:13.418155 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:21:13.418155 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 607/1217] 10.222.0.102 () {56 vars in 1116 bytes} [Thu Sep 1 17:21:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:21:13.422739 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd782ea-f890-4953-b068-508970f6977f None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:21:13.549317 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd782ea-f890-4953-b068-508970f6977f None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:21:13.578450 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:21:13.593110 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195b4bac-cc01-49ec-ba2b-a48d79a2be0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:21:13.593874 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195b4bac-cc01-49ec-ba2b-a48d79a2be0b None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:21:13.594248 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195b4bac-cc01-49ec-ba2b-a48d79a2be0b None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:21:13.594613 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195b4bac-cc01-49ec-ba2b-a48d79a2be0b None admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:21:13.595398 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195b4bac-cc01-49ec-ba2b-a48d79a2be0b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:21:13.595799 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195b4bac-cc01-49ec-ba2b-a48d79a2be0b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:21:13.597613 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195b4bac-cc01-49ec-ba2b-a48d79a2be0b None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:21:13.614081 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 611/1218] 10.222.0.102 () {56 vars in 1153 bytes} [Thu Sep 1 17:21:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:21:13.620552 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afd8c66-eb85-4d73-84a7-a7b1139ab73c None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:21:13.743580 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afd8c66-eb85-4d73-84a7-a7b1139ab73c None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:21:13.781354 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:21:13.788611 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9bda5c-d687-4693-919e-45ba69a7cab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:21:13.790038 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9bda5c-d687-4693-919e-45ba69a7cab3 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:21:13.790424 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9bda5c-d687-4693-919e-45ba69a7cab3 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:21:13.790790 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9bda5c-d687-4693-919e-45ba69a7cab3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:21:13.791663 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9bda5c-d687-4693-919e-45ba69a7cab3 None admin] RBAC: Adding query filter params (role.id=f21e23654e4740158c7d4f46e726e515, scope.project.id=e78fff7f1f6e4f68b8d233ad2e5bbd4f, user.id=134ef6039cef468bae0e3f3be79b8073) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:21:13.792067 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9bda5c-d687-4693-919e-45ba69a7cab3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:21:13.793877 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9bda5c-d687-4693-919e-45ba69a7cab3 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:21:13.800409 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 608/1219] 10.222.0.102 () {56 vars in 1379 bytes} [Thu Sep 1 17:21:13 2022] GET /identity/v3/role_assignments?user.id=134ef6039cef468bae0e3f3be79b8073&scope.project.id=e78fff7f1f6e4f68b8d233ad2e5bbd4f&role.id=f21e23654e4740158c7d4f46e726e515 => generated 260 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:21:16.034686 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f12ec45-b481-4800-9d5c-5a0baeafffe3 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:21:16.035258 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f12ec45-b481-4800-9d5c-5a0baeafffe3 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:21:16.035402 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f12ec45-b481-4800-9d5c-5a0baeafffe3 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:21:16.036864 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 612/1220] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:21:16 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:21:16.042649 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260677c4-46f6-4757-a00e-354096730d56 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:21:16.043096 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260677c4-46f6-4757-a00e-354096730d56 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:21:16.043237 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260677c4-46f6-4757-a00e-354096730d56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:21:16.061840 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-260677c4-46f6-4757-a00e-354096730d56 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:21:16.063027 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-260677c4-46f6-4757-a00e-354096730d56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:21:16.138008 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 609/1221] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:21:16 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:21:16.475556 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f12ec45-b481-4800-9d5c-5a0baeafffe3 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:21:16.587847 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f12ec45-b481-4800-9d5c-5a0baeafffe3 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:21:16.626744 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14287ec-07e1-4219-b5b2-80641bb84559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:21:16.627447 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14287ec-07e1-4219-b5b2-80641bb84559 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:21:16.627747 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14287ec-07e1-4219-b5b2-80641bb84559 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:21:16.628785 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14287ec-07e1-4219-b5b2-80641bb84559 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:21:16.640454 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14287ec-07e1-4219-b5b2-80641bb84559 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:21:16.645326 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-a14287ec-07e1-4219-b5b2-80641bb84559 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:21:16.645326 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:21:16.645326 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:21:16.645326 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:21:16.645326 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:21:16.645326 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:21:16.645326 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:21:16.645326 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:21:16.645326 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:21:16.645326 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:21:16.645326 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:21:16.645326 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:21:16.645326 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:21:16.645326 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:21:16.645326 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:21:16.645326 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:21:16.646531 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:21:16.646531 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:21:16.646531 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:21:16.646531 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:21:16.646531 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:21:16.646531 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:21:16.646531 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:21:16.646531 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:21:16.646531 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:21:16.646531 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:21:16.646531 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:21:16.646531 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:21:16.646531 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:21:16.646531 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:21:16.646531 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:21:16.646531 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:21:16.647585 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:21:16.647585 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:21:16.647585 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:21:16.647585 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:21:16.647585 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:21:16.647585 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:21:16.647585 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:21:16.647585 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:21:16.647585 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:21:16.647585 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:21:16.647585 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:21:16.647585 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:21:16.647585 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 613/1222] 10.222.0.102 () {56 vars in 1107 bytes} [Thu Sep 1 17:21:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:21:16.655341 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260677c4-46f6-4757-a00e-354096730d56 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:21:16.777397 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260677c4-46f6-4757-a00e-354096730d56 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:21:16.805923 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:21:16.821638 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0259a7f7-3f8f-47a1-813e-e36721f126f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:21:16.822253 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0259a7f7-3f8f-47a1-813e-e36721f126f8 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:21:16.822528 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0259a7f7-3f8f-47a1-813e-e36721f126f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:21:16.822798 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0259a7f7-3f8f-47a1-813e-e36721f126f8 None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:21:16.823460 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0259a7f7-3f8f-47a1-813e-e36721f126f8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:21:16.823757 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0259a7f7-3f8f-47a1-813e-e36721f126f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:21:16.825163 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0259a7f7-3f8f-47a1-813e-e36721f126f8 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:21:16.833816 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 610/1223] 10.222.0.102 () {56 vars in 1108 bytes} [Thu Sep 1 17:21:16 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:21:16.840606 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14287ec-07e1-4219-b5b2-80641bb84559 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:21:16.893091 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:21:16.958476 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14287ec-07e1-4219-b5b2-80641bb84559 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:21:16.998621 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628f5b3a-18d8-4efc-aa50-c8bf68d23be4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:21:16.999317 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628f5b3a-18d8-4efc-aa50-c8bf68d23be4 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:21:16.999702 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628f5b3a-18d8-4efc-aa50-c8bf68d23be4 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:21:17.000086 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628f5b3a-18d8-4efc-aa50-c8bf68d23be4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:21:17.015168 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628f5b3a-18d8-4efc-aa50-c8bf68d23be4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:21:17.016972 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628f5b3a-18d8-4efc-aa50-c8bf68d23be4 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:21:17.034067 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-628f5b3a-18d8-4efc-aa50-c8bf68d23be4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:21:17.034067 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:21:17.034067 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:21:17.034067 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:21:17.034067 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:21:17.034067 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:21:17.034067 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:21:17.034067 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:21:17.034067 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:21:17.034067 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:21:17.034067 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:21:17.034067 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:21:17.034067 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:21:17.034067 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:21:17.034067 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:21:17.034067 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:21:17.040561 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:21:17.040561 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:21:17.040561 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:21:17.040561 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:21:17.040561 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:21:17.040561 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:21:17.040561 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:21:17.040561 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:21:17.040561 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:21:17.040561 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:21:17.040561 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:21:17.040561 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:21:17.040561 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:21:17.040561 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:21:17.040561 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:21:17.040561 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:21:17.040561 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:21:17.041501 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:21:17.041501 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:21:17.041501 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:21:17.041501 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 614/1224] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:21:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:21:17.041740 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0259a7f7-3f8f-47a1-813e-e36721f126f8 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:21:17.154286 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0259a7f7-3f8f-47a1-813e-e36721f126f8 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:21:17.191399 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171daf0d-b957-45f1-9a88-e6fd556c750e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:21:17.192105 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171daf0d-b957-45f1-9a88-e6fd556c750e None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:21:17.192387 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171daf0d-b957-45f1-9a88-e6fd556c750e None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:21:17.192636 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171daf0d-b957-45f1-9a88-e6fd556c750e None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:21:17.193323 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171daf0d-b957-45f1-9a88-e6fd556c750e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:21:17.193650 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171daf0d-b957-45f1-9a88-e6fd556c750e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:21:17.195236 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171daf0d-b957-45f1-9a88-e6fd556c750e None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:21:17.210963 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 611/1225] 10.222.0.102 () {56 vars in 1114 bytes} [Thu Sep 1 17:21:17 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:21:17.215870 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628f5b3a-18d8-4efc-aa50-c8bf68d23be4 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:21:17.361534 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:21:17.373525 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628f5b3a-18d8-4efc-aa50-c8bf68d23be4 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:21:17.420395 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7486e69-96c3-4854-aaca-2b49741ffba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:21:17.421049 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7486e69-96c3-4854-aaca-2b49741ffba7 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:21:17.421347 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7486e69-96c3-4854-aaca-2b49741ffba7 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:21:17.421627 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7486e69-96c3-4854-aaca-2b49741ffba7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:21:17.431027 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7486e69-96c3-4854-aaca-2b49741ffba7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:21:17.432570 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7486e69-96c3-4854-aaca-2b49741ffba7 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:21:17.446311 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-a7486e69-96c3-4854-aaca-2b49741ffba7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 01 17:21:17.446311 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:21:17.446311 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:21:17.446311 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:21:17.446311 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:21:17.446311 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:21:17.446311 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:21:17.446311 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:21:17.446311 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:21:17.446311 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:21:17.446311 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:21:17.446311 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:21:17.446311 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:21:17.446311 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:21:17.446311 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:21:17.446311 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:21:17.448355 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:21:17.448355 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:21:17.448355 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:21:17.448355 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:21:17.448355 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:21:17.448355 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:21:17.448355 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:21:17.448355 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:21:17.448355 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:21:17.448355 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:21:17.448355 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:21:17.448355 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:21:17.448355 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:21:17.448355 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:21:17.448355 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:21:17.448355 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:21:17.448355 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:21:17.448355 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:21:17.450471 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:21:17.450471 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:21:17.450471 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:21:17.450471 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:21:17.450471 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:21:17.450471 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:21:17.450471 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:21:17.450471 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:21:17.450471 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 01 17:21:17.450471 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:21:17.450471 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 615/1226] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:21:17 2022] GET /identity/v3/users/placement => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:21:17.453911 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171daf0d-b957-45f1-9a88-e6fd556c750e None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:21:17.510843 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:21:17.569123 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171daf0d-b957-45f1-9a88-e6fd556c750e None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:21:17.608900 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798281b2-1099-4a2b-a47b-abfada642dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:21:17.609545 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798281b2-1099-4a2b-a47b-abfada642dbc None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:21:17.609827 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798281b2-1099-4a2b-a47b-abfada642dbc None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:21:17.610097 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798281b2-1099-4a2b-a47b-abfada642dbc None admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:21:17.610847 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798281b2-1099-4a2b-a47b-abfada642dbc None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:21:17.611151 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798281b2-1099-4a2b-a47b-abfada642dbc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:21:17.612917 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798281b2-1099-4a2b-a47b-abfada642dbc None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:21:17.635678 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 612/1227] 10.222.0.102 () {56 vars in 1148 bytes} [Thu Sep 1 17:21:17 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 380 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:21:17.641847 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7486e69-96c3-4854-aaca-2b49741ffba7 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:21:17.754190 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7486e69-96c3-4854-aaca-2b49741ffba7 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:21:17.791647 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb8da00-877e-494a-9579-428a9da109a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:21:17.792271 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb8da00-877e-494a-9579-428a9da109a8 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:21:17.792556 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb8da00-877e-494a-9579-428a9da109a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:21:17.792826 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb8da00-877e-494a-9579-428a9da109a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:21:17.804335 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb8da00-877e-494a-9579-428a9da109a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:21:17.805872 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb8da00-877e-494a-9579-428a9da109a8 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:21:17.820641 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-dcb8da00-877e-494a-9579-428a9da109a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:21:17.820641 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:21:17.820641 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:21:17.820641 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:21:17.820641 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:21:17.820641 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:21:17.820641 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:21:17.820641 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:21:17.820641 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:21:17.820641 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:21:17.820641 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:21:17.820641 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:21:17.820641 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:21:17.820641 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:21:17.820641 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:21:17.820641 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:21:17.821860 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:21:17.821860 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:21:17.821860 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:21:17.821860 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:21:17.821860 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:21:17.821860 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:21:17.821860 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:21:17.821860 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:21:17.821860 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:21:17.821860 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:21:17.821860 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:21:17.821860 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:21:17.821860 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:21:17.821860 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:21:17.821860 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:21:17.821860 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:21:17.821860 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:21:17.822979 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:21:17.822979 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:21:17.822979 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:21:17.822979 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 616/1228] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:21:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:21:17.828043 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798281b2-1099-4a2b-a47b-abfada642dbc None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:21:17.935780 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798281b2-1099-4a2b-a47b-abfada642dbc None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:21:17.938202 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:21:17.975547 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b6e996-bc27-40de-9225-a68258db8855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:21:17.976291 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b6e996-bc27-40de-9225-a68258db8855 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:21:17.976648 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b6e996-bc27-40de-9225-a68258db8855 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:21:17.977535 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b6e996-bc27-40de-9225-a68258db8855 None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:21:17.978371 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b6e996-bc27-40de-9225-a68258db8855 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:21:17.978687 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b6e996-bc27-40de-9225-a68258db8855 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:21:17.980143 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b6e996-bc27-40de-9225-a68258db8855 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:21:17.992538 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 613/1229] 10.222.0.102 () {56 vars in 1114 bytes} [Thu Sep 1 17:21:17 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:21:17.998931 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb8da00-877e-494a-9579-428a9da109a8 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:21:18.011851 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:21:18.125077 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb8da00-877e-494a-9579-428a9da109a8 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:21:18.162287 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bf332e-6d2b-4995-871a-ea790bb93cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:21:18.162908 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bf332e-6d2b-4995-871a-ea790bb93cfd None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:21:18.163651 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bf332e-6d2b-4995-871a-ea790bb93cfd None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:21:18.163651 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bf332e-6d2b-4995-871a-ea790bb93cfd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:21:18.170251 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bf332e-6d2b-4995-871a-ea790bb93cfd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:21:18.171681 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bf332e-6d2b-4995-871a-ea790bb93cfd None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:21:18.183397 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-d4bf332e-6d2b-4995-871a-ea790bb93cfd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:21:18.183397 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:21:18.183397 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:21:18.183397 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:21:18.183397 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:21:18.183397 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:21:18.183397 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:21:18.183397 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:21:18.183397 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:21:18.183397 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:21:18.183397 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:21:18.183397 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:21:18.183397 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:21:18.183397 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:21:18.183397 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:21:18.184471 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:21:18.184471 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:21:18.184471 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:21:18.184471 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:21:18.184471 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:21:18.184471 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:21:18.184471 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:21:18.184471 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:21:18.184471 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:21:18.184471 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:21:18.184471 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:21:18.184471 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:21:18.184471 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:21:18.184471 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:21:18.184471 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:21:18.184471 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:21:18.184471 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:21:18.185599 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:21:18.185599 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:21:18.185599 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:21:18.185599 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:21:18.185599 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:21:18.185599 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:21:18.185599 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:21:18.185599 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:21:18.185599 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 617/1230] 10.222.0.102 () {56 vars in 1116 bytes} [Thu Sep 1 17:21:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:21:18.190861 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b6e996-bc27-40de-9225-a68258db8855 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:21:18.308138 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b6e996-bc27-40de-9225-a68258db8855 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:21:18.352490 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a4e7b6-2798-4ad7-8698-bd8d48053b4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:21:18.353110 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a4e7b6-2798-4ad7-8698-bd8d48053b4f None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:21:18.353417 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a4e7b6-2798-4ad7-8698-bd8d48053b4f None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:21:18.353695 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a4e7b6-2798-4ad7-8698-bd8d48053b4f None admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:21:18.354394 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a4e7b6-2798-4ad7-8698-bd8d48053b4f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:21:18.354701 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a4e7b6-2798-4ad7-8698-bd8d48053b4f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:21:18.356287 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a4e7b6-2798-4ad7-8698-bd8d48053b4f None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:21:18.373279 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 614/1231] 10.222.0.102 () {56 vars in 1153 bytes} [Thu Sep 1 17:21:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:21:18.379356 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bf332e-6d2b-4995-871a-ea790bb93cfd None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:21:18.454668 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:21:18.498597 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bf332e-6d2b-4995-871a-ea790bb93cfd None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:21:18.535953 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9347b180-8fb2-4354-a017-4f7db5551aa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:21:18.536618 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9347b180-8fb2-4354-a017-4f7db5551aa1 None admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:21:18.536898 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9347b180-8fb2-4354-a017-4f7db5551aa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:21:18.537173 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9347b180-8fb2-4354-a017-4f7db5551aa1 None admin] PATH_INFO: `/v3/projects/e78fff7f1f6e4f68b8d233ad2e5bbd4f/users/134ef6039cef468bae0e3f3be79b8073/roles/f21e23654e4740158c7d4f46e726e515` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:21:18.553701 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9347b180-8fb2-4354-a017-4f7db5551aa1 None admin] RBAC: Authorizing `identity:create_grant(project_id=e78fff7f1f6e4f68b8d233ad2e5bbd4f, user_id=134ef6039cef468bae0e3f3be79b8073, role_id=f21e23654e4740158c7d4f46e726e515)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:21:18.555681 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9347b180-8fb2-4354-a017-4f7db5551aa1 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:21:18.570811 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 618/1232] 10.222.0.102 () {58 vars in 1444 bytes} [Thu Sep 1 17:21:18 2022] PUT /identity/v3/projects/e78fff7f1f6e4f68b8d233ad2e5bbd4f/users/134ef6039cef468bae0e3f3be79b8073/roles/f21e23654e4740158c7d4f46e726e515 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:21:20.594247 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21dfc6f-d0bd-498c-a52b-47310c30111c None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:21:20.594779 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21dfc6f-d0bd-498c-a52b-47310c30111c None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:21:20.594779 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21dfc6f-d0bd-498c-a52b-47310c30111c None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:21:20.596132 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 615/1233] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:21:20 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:21:20.602345 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d533fab-c81f-487a-87ff-fcfadd8fc919 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:21:20.602879 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d533fab-c81f-487a-87ff-fcfadd8fc919 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:21:20.602879 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d533fab-c81f-487a-87ff-fcfadd8fc919 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:21:20.621336 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-8d533fab-c81f-487a-87ff-fcfadd8fc919 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:21:20.623345 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-8d533fab-c81f-487a-87ff-fcfadd8fc919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:21:20.692711 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 619/1234] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:21:20 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:21:20.972381 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21dfc6f-d0bd-498c-a52b-47310c30111c None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:21:21.000193 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:21:21.102000 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21dfc6f-d0bd-498c-a52b-47310c30111c None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:21:21.137164 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca73ba9a-2eb7-4c8e-8ccc-cb8745cad942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:21:21.137862 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca73ba9a-2eb7-4c8e-8ccc-cb8745cad942 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:21:21.138145 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca73ba9a-2eb7-4c8e-8ccc-cb8745cad942 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:21:21.138417 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca73ba9a-2eb7-4c8e-8ccc-cb8745cad942 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:21:21.149163 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca73ba9a-2eb7-4c8e-8ccc-cb8745cad942 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:21:21.153974 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-ca73ba9a-2eb7-4c8e-8ccc-cb8745cad942 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:21:21.153974 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:21:21.153974 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:21:21.153974 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:21:21.153974 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:21:21.153974 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:21:21.153974 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:21:21.153974 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:21:21.153974 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:21:21.153974 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:21:21.153974 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:21:21.153974 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:21:21.153974 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:21:21.153974 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:21:21.153974 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:21:21.153974 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:21:21.155085 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:21:21.155085 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:21:21.155085 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:21:21.155085 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:21:21.155085 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:21:21.155085 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:21:21.155085 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:21:21.155085 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:21:21.155085 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:21:21.155085 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:21:21.155085 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:21:21.155085 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:21:21.155085 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:21:21.155085 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:21:21.155085 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:21:21.155085 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:21:21.155981 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:21:21.155981 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:21:21.155981 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:21:21.155981 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:21:21.155981 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:21:21.155981 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:21:21.155981 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:21:21.155981 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:21:21.155981 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:21:21.155981 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:21:21.155981 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:21:21.155981 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:21:21.155981 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 616/1235] 10.222.0.102 () {56 vars in 1107 bytes} [Thu Sep 1 17:21:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:21:21.161550 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d533fab-c81f-487a-87ff-fcfadd8fc919 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:21:21.210879 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:21:21.272480 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d533fab-c81f-487a-87ff-fcfadd8fc919 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:21:21.312045 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b14c70-ccde-4154-930a-638a6cdb7414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:21:21.312730 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b14c70-ccde-4154-930a-638a6cdb7414 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:21:21.313010 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b14c70-ccde-4154-930a-638a6cdb7414 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:21:21.313285 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b14c70-ccde-4154-930a-638a6cdb7414 None admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:21:21.313977 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b14c70-ccde-4154-930a-638a6cdb7414 None admin] RBAC: Adding query filter params (name=service) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:21:21.314320 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b14c70-ccde-4154-930a-638a6cdb7414 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:21:21.315807 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b14c70-ccde-4154-930a-638a6cdb7414 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:21:21.324250 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 620/1236] 10.222.0.102 () {56 vars in 1108 bytes} [Thu Sep 1 17:21:21 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:21:21.330694 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca73ba9a-2eb7-4c8e-8ccc-cb8745cad942 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:21:21.433410 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:21:21.451124 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca73ba9a-2eb7-4c8e-8ccc-cb8745cad942 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:21:21.489391 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2759b28b-44ec-4b50-a2f5-33fbad36671c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:21:21.490051 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2759b28b-44ec-4b50-a2f5-33fbad36671c None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:21:21.490355 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2759b28b-44ec-4b50-a2f5-33fbad36671c None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:21:21.490669 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2759b28b-44ec-4b50-a2f5-33fbad36671c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:21:21.502939 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2759b28b-44ec-4b50-a2f5-33fbad36671c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:21:21.504402 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2759b28b-44ec-4b50-a2f5-33fbad36671c None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:21:21.520715 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-2759b28b-44ec-4b50-a2f5-33fbad36671c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:21:21.520715 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:21:21.520715 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:21:21.520715 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:21:21.520715 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:21:21.520715 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:21:21.520715 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:21:21.520715 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:21:21.520715 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:21:21.520715 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:21:21.520715 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:21:21.520715 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:21:21.520715 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:21:21.520715 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:21:21.520715 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:21:21.520715 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:21:21.521779 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:21:21.521779 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:21:21.521779 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:21:21.521779 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:21:21.521779 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:21:21.521779 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:21:21.521779 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:21:21.521779 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:21:21.521779 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:21:21.521779 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:21:21.521779 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:21:21.521779 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:21:21.521779 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:21:21.521779 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:21:21.521779 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:21:21.521779 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:21:21.521779 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:21:21.522898 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:21:21.522898 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:21:21.522898 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:21:21.522898 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 617/1237] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:21:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:21:21.528998 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b14c70-ccde-4154-930a-638a6cdb7414 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:21:21.639395 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b14c70-ccde-4154-930a-638a6cdb7414 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:21:21.649618 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:21:21.677444 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27ebe84-b676-4649-8227-3a1bd2850e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:21:21.678166 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27ebe84-b676-4649-8227-3a1bd2850e6b None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:21:21.678445 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27ebe84-b676-4649-8227-3a1bd2850e6b None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:21:21.678735 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27ebe84-b676-4649-8227-3a1bd2850e6b None admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:21:21.679462 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27ebe84-b676-4649-8227-3a1bd2850e6b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:21:21.679814 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27ebe84-b676-4649-8227-3a1bd2850e6b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:21:21.681387 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27ebe84-b676-4649-8227-3a1bd2850e6b None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:21:21.694534 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 621/1238] 10.222.0.102 () {56 vars in 1114 bytes} [Thu Sep 1 17:21:21 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:21:21.700624 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2759b28b-44ec-4b50-a2f5-33fbad36671c None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:21:21.838941 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2759b28b-44ec-4b50-a2f5-33fbad36671c None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:21:21.881443 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d68446-8a61-4147-8b94-ad6c6a40dd83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:21:21.882254 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d68446-8a61-4147-8b94-ad6c6a40dd83 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:21:21.883073 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d68446-8a61-4147-8b94-ad6c6a40dd83 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:21:21.883383 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d68446-8a61-4147-8b94-ad6c6a40dd83 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:21:21.887371 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:21:21.893696 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d68446-8a61-4147-8b94-ad6c6a40dd83 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:21:21.895306 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d68446-8a61-4147-8b94-ad6c6a40dd83 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:21:21.908365 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-a7d68446-8a61-4147-8b94-ad6c6a40dd83 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 01 17:21:21.908365 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:21:21.908365 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:21:21.908365 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:21:21.908365 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:21:21.908365 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:21:21.908365 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:21:21.908365 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:21:21.908365 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:21:21.908365 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:21:21.908365 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:21:21.908365 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:21:21.908365 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:21:21.908365 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:21:21.908365 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:21:21.908365 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:21:21.909531 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:21:21.909531 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:21:21.909531 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:21:21.909531 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:21:21.909531 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:21:21.909531 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:21:21.909531 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:21:21.909531 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:21:21.909531 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:21:21.909531 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:21:21.909531 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:21:21.909531 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:21:21.909531 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:21:21.909531 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:21:21.909531 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:21:21.909531 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:21:21.909531 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:21:21.909531 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:21:21.910507 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:21:21.910507 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:21:21.910507 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:21:21.910507 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:21:21.910507 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:21:21.910507 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:21:21.910507 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:21:21.910507 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:21:21.910507 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 01 17:21:21.910507 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:21:21.910507 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 618/1239] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:21:21 2022] GET /identity/v3/users/placement => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:21:21.915697 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27ebe84-b676-4649-8227-3a1bd2850e6b None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:21:22.028186 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27ebe84-b676-4649-8227-3a1bd2850e6b None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:21:22.065293 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24424a25-4e2b-4ace-9aa7-f06075463475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:21:22.066054 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24424a25-4e2b-4ace-9aa7-f06075463475 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:21:22.066346 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24424a25-4e2b-4ace-9aa7-f06075463475 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:21:22.066607 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24424a25-4e2b-4ace-9aa7-f06075463475 None admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:21:22.067542 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24424a25-4e2b-4ace-9aa7-f06075463475 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:21:22.067991 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24424a25-4e2b-4ace-9aa7-f06075463475 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:21:22.069612 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24424a25-4e2b-4ace-9aa7-f06075463475 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:21:22.095861 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 622/1240] 10.222.0.102 () {56 vars in 1148 bytes} [Thu Sep 1 17:21:21 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 380 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:21:22.104834 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d68446-8a61-4147-8b94-ad6c6a40dd83 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:21:22.230309 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d68446-8a61-4147-8b94-ad6c6a40dd83 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:21:22.272660 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90646fa-a8b0-49f6-9061-8bfb81470298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:21:22.272966 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90646fa-a8b0-49f6-9061-8bfb81470298 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:21:22.273754 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90646fa-a8b0-49f6-9061-8bfb81470298 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:21:22.274070 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90646fa-a8b0-49f6-9061-8bfb81470298 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:21:22.342574 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:21:22.350715 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90646fa-a8b0-49f6-9061-8bfb81470298 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:21:22.352915 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90646fa-a8b0-49f6-9061-8bfb81470298 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:21:22.476244 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 619/1241] 10.222.0.102 () {58 vars in 1287 bytes} [Thu Sep 1 17:21:22 2022] GET /identity/v3/auth/tokens => generated 2123 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 17:21:22.483094 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24424a25-4e2b-4ace-9aa7-f06075463475 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:21:22.514690 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:21:22.590980 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24424a25-4e2b-4ace-9aa7-f06075463475 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:21:22.626800 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a2ecfb-dcc5-48d7-aa4b-a7ea03bc6ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:21:22.627353 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a2ecfb-dcc5-48d7-aa4b-a7ea03bc6ce7 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:21:22.627632 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a2ecfb-dcc5-48d7-aa4b-a7ea03bc6ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:21:22.627905 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a2ecfb-dcc5-48d7-aa4b-a7ea03bc6ce7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:21:22.638474 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a2ecfb-dcc5-48d7-aa4b-a7ea03bc6ce7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:21:22.641041 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a2ecfb-dcc5-48d7-aa4b-a7ea03bc6ce7 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:21:22.658958 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-81a2ecfb-dcc5-48d7-aa4b-a7ea03bc6ce7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:21:22.658958 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:21:22.658958 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:21:22.658958 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:21:22.658958 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:21:22.658958 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:21:22.658958 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:21:22.658958 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:21:22.658958 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:21:22.658958 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:21:22.658958 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:21:22.658958 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:21:22.658958 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:21:22.658958 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:21:22.658958 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:21:22.658958 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:21:22.660109 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:21:22.660109 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:21:22.660109 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:21:22.660109 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:21:22.660109 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:21:22.660109 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:21:22.660109 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:21:22.660109 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:21:22.660109 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:21:22.660109 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:21:22.660109 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:21:22.660109 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:21:22.660109 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:21:22.660109 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:21:22.660109 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:21:22.660109 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:21:22.660109 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:21:22.661122 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:21:22.661122 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:21:22.661122 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:21:22.661122 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 623/1242] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:21:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:21:22.666610 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90646fa-a8b0-49f6-9061-8bfb81470298 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:21:22.774666 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:21:22.782134 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90646fa-a8b0-49f6-9061-8bfb81470298 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:21:22.817677 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7aac041-9e9c-4817-b212-e0b695d38913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:21:22.818289 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aac041-9e9c-4817-b212-e0b695d38913 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:21:22.818566 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aac041-9e9c-4817-b212-e0b695d38913 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:21:22.818837 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aac041-9e9c-4817-b212-e0b695d38913 None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:21:22.819476 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7aac041-9e9c-4817-b212-e0b695d38913 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:21:22.819782 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7aac041-9e9c-4817-b212-e0b695d38913 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:21:22.821676 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7aac041-9e9c-4817-b212-e0b695d38913 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:21:22.834795 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 620/1243] 10.222.0.102 () {56 vars in 1114 bytes} [Thu Sep 1 17:21:22 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:21:22.841255 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a2ecfb-dcc5-48d7-aa4b-a7ea03bc6ce7 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:21:22.931819 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:21:22.956791 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a2ecfb-dcc5-48d7-aa4b-a7ea03bc6ce7 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:21:22.994207 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c755679f-49da-414d-83e6-b858be97d408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:21:22.995008 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c755679f-49da-414d-83e6-b858be97d408 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:21:22.995380 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c755679f-49da-414d-83e6-b858be97d408 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:21:22.995676 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c755679f-49da-414d-83e6-b858be97d408 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:21:23.003600 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c755679f-49da-414d-83e6-b858be97d408 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:21:23.005316 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c755679f-49da-414d-83e6-b858be97d408 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:21:23.019565 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-c755679f-49da-414d-83e6-b858be97d408 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:21:23.019565 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:21:23.019565 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:21:23.019565 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:21:23.019565 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:21:23.019565 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:21:23.019565 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:21:23.019565 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:21:23.019565 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:21:23.019565 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:21:23.019565 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:21:23.019565 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:21:23.019565 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:21:23.019565 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:21:23.019565 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:21:23.020746 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:21:23.020746 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:21:23.020746 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:21:23.020746 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:21:23.020746 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:21:23.020746 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:21:23.020746 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:21:23.020746 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:21:23.020746 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:21:23.020746 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:21:23.020746 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:21:23.020746 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:21:23.020746 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:21:23.020746 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:21:23.020746 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:21:23.020746 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:21:23.020746 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:21:23.021730 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:21:23.021730 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:21:23.021730 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:21:23.021730 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:21:23.021730 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:21:23.021730 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:21:23.021730 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:21:23.021730 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:21:23.021730 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 624/1244] 10.222.0.102 () {56 vars in 1116 bytes} [Thu Sep 1 17:21:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:21:23.026850 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7aac041-9e9c-4817-b212-e0b695d38913 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:21:23.146500 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7aac041-9e9c-4817-b212-e0b695d38913 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:21:23.183784 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1ceb24-edc7-4756-b734-0b3849b71c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:21:23.184243 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1ceb24-edc7-4756-b734-0b3849b71c12 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:21:23.184546 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1ceb24-edc7-4756-b734-0b3849b71c12 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:21:23.184830 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1ceb24-edc7-4756-b734-0b3849b71c12 None admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:21:23.185624 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1ceb24-edc7-4756-b734-0b3849b71c12 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:21:23.185934 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1ceb24-edc7-4756-b734-0b3849b71c12 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:21:23.187492 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1ceb24-edc7-4756-b734-0b3849b71c12 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:21:23.200445 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 621/1245] 10.222.0.102 () {56 vars in 1153 bytes} [Thu Sep 1 17:21:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:21:23.207245 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c755679f-49da-414d-83e6-b858be97d408 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:21:23.335507 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c755679f-49da-414d-83e6-b858be97d408 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:21:23.372508 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46011fa-d739-492a-934e-4400644a7b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:21:23.373193 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46011fa-d739-492a-934e-4400644a7b69 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:21:23.373553 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46011fa-d739-492a-934e-4400644a7b69 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:21:23.373853 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46011fa-d739-492a-934e-4400644a7b69 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:21:23.374656 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46011fa-d739-492a-934e-4400644a7b69 None admin] RBAC: Adding query filter params (role.id=f21e23654e4740158c7d4f46e726e515, scope.project.id=e78fff7f1f6e4f68b8d233ad2e5bbd4f, user.id=134ef6039cef468bae0e3f3be79b8073) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:21:23.375011 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46011fa-d739-492a-934e-4400644a7b69 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:21:23.376624 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46011fa-d739-492a-934e-4400644a7b69 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:21:23.383317 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 625/1246] 10.222.0.102 () {56 vars in 1379 bytes} [Thu Sep 1 17:21:23 2022] GET /identity/v3/role_assignments?user.id=134ef6039cef468bae0e3f3be79b8073&scope.project.id=e78fff7f1f6e4f68b8d233ad2e5bbd4f&role.id=f21e23654e4740158c7d4f46e726e515 => generated 610 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:21:25.470328 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01766de-15aa-4887-8986-58ceabbb4777 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:21:25.470964 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01766de-15aa-4887-8986-58ceabbb4777 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:21:25.470964 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01766de-15aa-4887-8986-58ceabbb4777 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:21:25.472481 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 622/1247] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:21:25 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:21:25.478475 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec04f4ee-655b-4da4-8df5-3e0420fc4025 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:21:25.480253 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec04f4ee-655b-4da4-8df5-3e0420fc4025 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:21:25.480253 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec04f4ee-655b-4da4-8df5-3e0420fc4025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:21:25.486865 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:21:25.501142 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-ec04f4ee-655b-4da4-8df5-3e0420fc4025 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:21:25.502551 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-ec04f4ee-655b-4da4-8df5-3e0420fc4025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:21:25.583697 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 626/1248] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:21:25 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:21:25.900451 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01766de-15aa-4887-8986-58ceabbb4777 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:21:25.922681 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:21:26.025693 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01766de-15aa-4887-8986-58ceabbb4777 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:21:26.066555 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c86690-ad55-484a-9564-446a405acf29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:21:26.067297 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c86690-ad55-484a-9564-446a405acf29 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:21:26.067642 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c86690-ad55-484a-9564-446a405acf29 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:21:26.068028 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c86690-ad55-484a-9564-446a405acf29 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:21:26.082088 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c86690-ad55-484a-9564-446a405acf29 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:21:26.088270 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-40c86690-ad55-484a-9564-446a405acf29 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:21:26.088270 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:21:26.088270 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:21:26.088270 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:21:26.088270 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:21:26.088270 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:21:26.088270 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:21:26.088270 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:21:26.088270 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:21:26.088270 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:21:26.088270 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:21:26.088270 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:21:26.088270 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:21:26.088270 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:21:26.088270 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:21:26.088270 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:21:26.089519 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:21:26.089519 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:21:26.089519 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:21:26.089519 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:21:26.089519 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:21:26.089519 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:21:26.089519 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:21:26.089519 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:21:26.089519 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:21:26.089519 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:21:26.089519 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:21:26.089519 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:21:26.089519 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:21:26.089519 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:21:26.089519 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:21:26.089519 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:21:26.090480 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:21:26.090480 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:21:26.090480 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:21:26.090480 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:21:26.090480 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:21:26.090480 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:21:26.090480 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:21:26.090480 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:21:26.090480 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:21:26.090480 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:21:26.090480 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:21:26.090480 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:21:26.090480 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 623/1249] 10.222.0.102 () {56 vars in 1101 bytes} [Thu Sep 1 17:21:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:21:26.096078 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec04f4ee-655b-4da4-8df5-3e0420fc4025 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:21:26.225672 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec04f4ee-655b-4da4-8df5-3e0420fc4025 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:21:26.267502 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:21:26.275656 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf8c7b1-335e-439e-b23d-5966313f4ac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:21:26.276695 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf8c7b1-335e-439e-b23d-5966313f4ac2 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:21:26.277145 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf8c7b1-335e-439e-b23d-5966313f4ac2 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:21:26.277693 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf8c7b1-335e-439e-b23d-5966313f4ac2 None admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:21:26.278794 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf8c7b1-335e-439e-b23d-5966313f4ac2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:21:26.279630 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf8c7b1-335e-439e-b23d-5966313f4ac2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:21:26.281472 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf8c7b1-335e-439e-b23d-5966313f4ac2 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:21:26.293611 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 627/1250] 10.222.0.102 () {56 vars in 1104 bytes} [Thu Sep 1 17:21:26 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:21:26.298680 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c86690-ad55-484a-9564-446a405acf29 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:21:26.381851 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:21:26.407134 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c86690-ad55-484a-9564-446a405acf29 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:21:26.449827 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b330437-b167-48e2-8346-a90ceaf803e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:21:26.450561 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b330437-b167-48e2-8346-a90ceaf803e2 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:21:26.450826 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b330437-b167-48e2-8346-a90ceaf803e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:21:26.451102 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b330437-b167-48e2-8346-a90ceaf803e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:21:26.462097 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b330437-b167-48e2-8346-a90ceaf803e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:21:26.463751 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b330437-b167-48e2-8346-a90ceaf803e2 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:21:26.481215 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-1b330437-b167-48e2-8346-a90ceaf803e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:21:26.481215 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:21:26.481215 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:21:26.481215 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:21:26.481215 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:21:26.481215 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:21:26.481215 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:21:26.481215 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:21:26.481215 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:21:26.481215 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:21:26.481215 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:21:26.481215 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:21:26.481215 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:21:26.481215 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:21:26.481215 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:21:26.481215 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:21:26.482809 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:21:26.482809 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:21:26.482809 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:21:26.482809 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:21:26.482809 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:21:26.482809 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:21:26.482809 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:21:26.482809 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:21:26.482809 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:21:26.482809 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:21:26.482809 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:21:26.482809 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:21:26.482809 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:21:26.482809 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:21:26.482809 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:21:26.482809 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:21:26.482809 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:21:26.483710 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:21:26.483710 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:21:26.483710 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:21:26.483710 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 624/1251] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:21:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:21:26.488446 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf8c7b1-335e-439e-b23d-5966313f4ac2 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:21:26.601918 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf8c7b1-335e-439e-b23d-5966313f4ac2 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:21:26.641400 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fa3daa-472e-4ed9-92ef-118674115894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:21:26.642105 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fa3daa-472e-4ed9-92ef-118674115894 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:21:26.642486 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fa3daa-472e-4ed9-92ef-118674115894 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:21:26.642876 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fa3daa-472e-4ed9-92ef-118674115894 None admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:21:26.643619 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fa3daa-472e-4ed9-92ef-118674115894 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:21:26.644023 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fa3daa-472e-4ed9-92ef-118674115894 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:21:26.645514 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fa3daa-472e-4ed9-92ef-118674115894 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:21:26.658054 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 628/1252] 10.222.0.102 () {56 vars in 1114 bytes} [Thu Sep 1 17:21:26 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:21:26.664486 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b330437-b167-48e2-8346-a90ceaf803e2 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:21:26.770431 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b330437-b167-48e2-8346-a90ceaf803e2 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:21:26.799507 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:21:26.813539 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efb092e-4aef-4a2c-b7cc-26757d42dc47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:21:26.814342 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efb092e-4aef-4a2c-b7cc-26757d42dc47 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:21:26.814701 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efb092e-4aef-4a2c-b7cc-26757d42dc47 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:21:26.815082 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efb092e-4aef-4a2c-b7cc-26757d42dc47 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:21:26.824885 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efb092e-4aef-4a2c-b7cc-26757d42dc47 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:21:26.826753 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efb092e-4aef-4a2c-b7cc-26757d42dc47 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:21:26.840370 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-4efb092e-4aef-4a2c-b7cc-26757d42dc47 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 01 17:21:26.840370 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:21:26.840370 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:21:26.840370 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:21:26.840370 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:21:26.840370 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:21:26.840370 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:21:26.840370 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:21:26.840370 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:21:26.840370 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:21:26.840370 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:21:26.840370 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:21:26.840370 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:21:26.840370 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:21:26.840370 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:21:26.840370 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:21:26.841614 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:21:26.841614 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:21:26.841614 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:21:26.841614 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:21:26.841614 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:21:26.841614 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:21:26.841614 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:21:26.841614 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:21:26.841614 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:21:26.841614 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:21:26.841614 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:21:26.841614 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:21:26.841614 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:21:26.841614 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:21:26.841614 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:21:26.841614 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:21:26.841614 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:21:26.841614 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:21:26.842539 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:21:26.842539 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:21:26.842539 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:21:26.842539 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:21:26.842539 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:21:26.842539 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:21:26.842539 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:21:26.842539 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:21:26.842539 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 01 17:21:26.842539 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:21:26.842539 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 625/1253] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:21:26 2022] GET /identity/v3/users/placement => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:21:26.848167 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fa3daa-472e-4ed9-92ef-118674115894 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:21:26.912260 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:21:26.964958 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fa3daa-472e-4ed9-92ef-118674115894 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:21:27.011129 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5609e3f2-85d3-46b7-a13c-40063bc007d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:21:27.011786 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5609e3f2-85d3-46b7-a13c-40063bc007d1 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:21:27.012089 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5609e3f2-85d3-46b7-a13c-40063bc007d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:21:27.012389 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5609e3f2-85d3-46b7-a13c-40063bc007d1 None admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:21:27.013228 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5609e3f2-85d3-46b7-a13c-40063bc007d1 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:21:27.013591 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5609e3f2-85d3-46b7-a13c-40063bc007d1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:21:27.015142 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5609e3f2-85d3-46b7-a13c-40063bc007d1 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:21:27.038403 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 629/1254] 10.222.0.102 () {56 vars in 1148 bytes} [Thu Sep 1 17:21:26 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 380 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:21:27.046033 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efb092e-4aef-4a2c-b7cc-26757d42dc47 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:21:27.164450 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efb092e-4aef-4a2c-b7cc-26757d42dc47 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:21:27.202142 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f3ae3c-97a7-4350-8256-8bc0ec9f926c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:21:27.202739 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f3ae3c-97a7-4350-8256-8bc0ec9f926c None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:21:27.203055 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f3ae3c-97a7-4350-8256-8bc0ec9f926c None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:21:27.203292 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f3ae3c-97a7-4350-8256-8bc0ec9f926c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:21:27.279195 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:21:27.301034 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f3ae3c-97a7-4350-8256-8bc0ec9f926c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:21:27.302630 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f3ae3c-97a7-4350-8256-8bc0ec9f926c None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:21:27.410740 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 626/1255] 10.222.0.102 () {58 vars in 1287 bytes} [Thu Sep 1 17:21:27 2022] GET /identity/v3/auth/tokens => generated 2123 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 17:21:27.416766 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5609e3f2-85d3-46b7-a13c-40063bc007d1 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:21:27.535188 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5609e3f2-85d3-46b7-a13c-40063bc007d1 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:21:27.551121 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:21:27.575928 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e1b2a8-8935-41ee-8e5a-8c0bc8938723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:21:27.576607 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e1b2a8-8935-41ee-8e5a-8c0bc8938723 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:21:27.576915 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e1b2a8-8935-41ee-8e5a-8c0bc8938723 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:21:27.577206 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e1b2a8-8935-41ee-8e5a-8c0bc8938723 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:21:27.587721 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e1b2a8-8935-41ee-8e5a-8c0bc8938723 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:21:27.589209 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e1b2a8-8935-41ee-8e5a-8c0bc8938723 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:21:27.604461 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-92e1b2a8-8935-41ee-8e5a-8c0bc8938723 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:21:27.604461 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:21:27.604461 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:21:27.604461 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:21:27.604461 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:21:27.604461 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:21:27.604461 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:21:27.604461 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:21:27.604461 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:21:27.604461 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:21:27.604461 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:21:27.604461 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:21:27.604461 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:21:27.604461 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:21:27.604461 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:21:27.604461 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:21:27.610441 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:21:27.610441 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:21:27.610441 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:21:27.610441 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:21:27.610441 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:21:27.610441 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:21:27.610441 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:21:27.610441 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:21:27.610441 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:21:27.610441 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:21:27.610441 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:21:27.610441 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:21:27.610441 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:21:27.610441 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:21:27.610441 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:21:27.610441 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:21:27.610441 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:21:27.611369 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:21:27.611369 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:21:27.611369 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:21:27.611369 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 630/1256] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:21:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:21:27.611600 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f3ae3c-97a7-4350-8256-8bc0ec9f926c None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:21:27.709102 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:21:27.722978 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f3ae3c-97a7-4350-8256-8bc0ec9f926c None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:21:27.761513 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7bfb35-e936-4771-90ab-92cf7589d008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:21:27.762117 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7bfb35-e936-4771-90ab-92cf7589d008 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:21:27.762407 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7bfb35-e936-4771-90ab-92cf7589d008 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:21:27.762703 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7bfb35-e936-4771-90ab-92cf7589d008 None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:21:27.763423 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7bfb35-e936-4771-90ab-92cf7589d008 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:21:27.763753 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7bfb35-e936-4771-90ab-92cf7589d008 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:21:27.765368 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7bfb35-e936-4771-90ab-92cf7589d008 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:21:27.778772 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 627/1257] 10.222.0.102 () {56 vars in 1114 bytes} [Thu Sep 1 17:21:27 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:21:27.785124 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e1b2a8-8935-41ee-8e5a-8c0bc8938723 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:21:27.898917 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e1b2a8-8935-41ee-8e5a-8c0bc8938723 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:21:27.934532 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b013e84e-e1c3-40bb-ae5b-ce4867868d13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:21:27.935161 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b013e84e-e1c3-40bb-ae5b-ce4867868d13 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:21:27.935480 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b013e84e-e1c3-40bb-ae5b-ce4867868d13 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:21:27.935760 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b013e84e-e1c3-40bb-ae5b-ce4867868d13 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:21:27.943402 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b013e84e-e1c3-40bb-ae5b-ce4867868d13 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:21:27.944918 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b013e84e-e1c3-40bb-ae5b-ce4867868d13 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:21:27.956938 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-b013e84e-e1c3-40bb-ae5b-ce4867868d13 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:21:27.956938 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:21:27.956938 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:21:27.956938 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:21:27.956938 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:21:27.956938 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:21:27.956938 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:21:27.956938 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:21:27.956938 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:21:27.956938 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:21:27.956938 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:21:27.956938 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:21:27.956938 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:21:27.956938 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:21:27.956938 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:21:27.957888 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:21:27.957888 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:21:27.957888 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:21:27.957888 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:21:27.957888 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:21:27.957888 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:21:27.957888 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:21:27.957888 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:21:27.957888 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:21:27.957888 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:21:27.957888 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:21:27.957888 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:21:27.957888 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:21:27.957888 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:21:27.957888 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:21:27.957888 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:21:27.957888 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:21:27.959211 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:21:27.959211 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:21:27.959211 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:21:27.959211 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:21:27.959211 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:21:27.959211 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:21:27.959211 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:21:27.959211 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:21:27.959211 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 631/1258] 10.222.0.102 () {56 vars in 1116 bytes} [Thu Sep 1 17:21:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:21:27.964044 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7bfb35-e936-4771-90ab-92cf7589d008 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:21:28.079003 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7bfb35-e936-4771-90ab-92cf7589d008 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:21:28.117107 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dabea3-1904-400a-b05f-853ef655e297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:21:28.117908 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dabea3-1904-400a-b05f-853ef655e297 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:21:28.118188 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dabea3-1904-400a-b05f-853ef655e297 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:21:28.118525 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dabea3-1904-400a-b05f-853ef655e297 None admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:21:28.119394 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dabea3-1904-400a-b05f-853ef655e297 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:21:28.119733 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dabea3-1904-400a-b05f-853ef655e297 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:21:28.121477 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dabea3-1904-400a-b05f-853ef655e297 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:21:28.137500 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 628/1259] 10.222.0.102 () {56 vars in 1153 bytes} [Thu Sep 1 17:21:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:21:28.144117 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b013e84e-e1c3-40bb-ae5b-ce4867868d13 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:21:28.174569 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:21:28.274868 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b013e84e-e1c3-40bb-ae5b-ce4867868d13 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:21:28.312044 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e66948-e4b7-4e78-8588-2ae10e35c511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:21:28.312505 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e66948-e4b7-4e78-8588-2ae10e35c511 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:21:28.312779 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e66948-e4b7-4e78-8588-2ae10e35c511 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:21:28.313045 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e66948-e4b7-4e78-8588-2ae10e35c511 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:21:28.313808 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e66948-e4b7-4e78-8588-2ae10e35c511 None admin] RBAC: Adding query filter params (role.id=36577a6f46d642d4b81805ede9872490, scope.project.id=e78fff7f1f6e4f68b8d233ad2e5bbd4f, user.id=134ef6039cef468bae0e3f3be79b8073) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:21:28.314112 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e66948-e4b7-4e78-8588-2ae10e35c511 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:21:28.315539 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e66948-e4b7-4e78-8588-2ae10e35c511 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:21:28.333354 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 632/1260] 10.222.0.102 () {56 vars in 1379 bytes} [Thu Sep 1 17:21:28 2022] GET /identity/v3/role_assignments?user.id=134ef6039cef468bae0e3f3be79b8073&scope.project.id=e78fff7f1f6e4f68b8d233ad2e5bbd4f&role.id=36577a6f46d642d4b81805ede9872490 => generated 260 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:21:30.211485 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336594cb-5f77-472c-88e1-6eae77634735 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:21:30.212007 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336594cb-5f77-472c-88e1-6eae77634735 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:21:30.212267 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336594cb-5f77-472c-88e1-6eae77634735 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:21:30.214559 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 629/1261] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:21:30 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:21:30.221276 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1830c1d3-584b-4635-9f91-c1e1e1a4a6ce None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:21:30.221834 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1830c1d3-584b-4635-9f91-c1e1e1a4a6ce None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:21:30.221996 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1830c1d3-584b-4635-9f91-c1e1e1a4a6ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:21:30.246393 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-1830c1d3-584b-4635-9f91-c1e1e1a4a6ce None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:21:30.248284 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-1830c1d3-584b-4635-9f91-c1e1e1a4a6ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:21:30.315632 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 633/1262] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:21:30 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:21:30.588054 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336594cb-5f77-472c-88e1-6eae77634735 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:21:30.599261 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:21:30.688353 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336594cb-5f77-472c-88e1-6eae77634735 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:21:30.723752 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13eb168-19dd-4974-a99b-5526a06a4275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:21:30.724397 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13eb168-19dd-4974-a99b-5526a06a4275 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:21:30.724704 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13eb168-19dd-4974-a99b-5526a06a4275 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:21:30.724942 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13eb168-19dd-4974-a99b-5526a06a4275 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:21:30.735817 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13eb168-19dd-4974-a99b-5526a06a4275 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:21:30.740526 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-d13eb168-19dd-4974-a99b-5526a06a4275 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:21:30.740526 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:21:30.740526 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:21:30.740526 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:21:30.740526 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:21:30.740526 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:21:30.740526 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:21:30.740526 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:21:30.740526 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:21:30.740526 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:21:30.740526 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:21:30.740526 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:21:30.740526 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:21:30.740526 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:21:30.740526 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:21:30.740526 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:21:30.741662 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:21:30.741662 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:21:30.741662 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:21:30.741662 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:21:30.741662 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:21:30.741662 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:21:30.741662 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:21:30.741662 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:21:30.741662 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:21:30.741662 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:21:30.741662 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:21:30.741662 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:21:30.741662 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:21:30.741662 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:21:30.741662 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:21:30.741662 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:21:30.742670 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:21:30.742670 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:21:30.742670 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:21:30.742670 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:21:30.742670 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:21:30.742670 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:21:30.742670 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:21:30.742670 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:21:30.742670 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:21:30.742670 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:21:30.742670 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:21:30.742670 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:21:30.742670 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 630/1263] 10.222.0.102 () {56 vars in 1101 bytes} [Thu Sep 1 17:21:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:21:30.747673 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1830c1d3-584b-4635-9f91-c1e1e1a4a6ce None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:21:30.766644 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:21:30.845575 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1830c1d3-584b-4635-9f91-c1e1e1a4a6ce None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:21:30.878318 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2127a894-dd25-4b01-a23c-c38637cf3e4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:21:30.878910 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2127a894-dd25-4b01-a23c-c38637cf3e4b None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:21:30.879187 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2127a894-dd25-4b01-a23c-c38637cf3e4b None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:21:30.879455 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2127a894-dd25-4b01-a23c-c38637cf3e4b None admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:21:30.880099 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2127a894-dd25-4b01-a23c-c38637cf3e4b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:21:30.880398 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2127a894-dd25-4b01-a23c-c38637cf3e4b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:21:30.882407 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2127a894-dd25-4b01-a23c-c38637cf3e4b None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:21:30.890134 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 634/1264] 10.222.0.102 () {56 vars in 1104 bytes} [Thu Sep 1 17:21:30 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:21:30.895988 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13eb168-19dd-4974-a99b-5526a06a4275 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:21:30.962792 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:21:30.997661 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13eb168-19dd-4974-a99b-5526a06a4275 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:21:31.031835 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1d1911-fa4d-4e3e-9c99-0a4fca42c580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:21:31.032482 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1d1911-fa4d-4e3e-9c99-0a4fca42c580 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:21:31.032758 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1d1911-fa4d-4e3e-9c99-0a4fca42c580 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:21:31.033033 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1d1911-fa4d-4e3e-9c99-0a4fca42c580 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:21:31.042951 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1d1911-fa4d-4e3e-9c99-0a4fca42c580 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:21:31.044397 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1d1911-fa4d-4e3e-9c99-0a4fca42c580 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:21:31.058576 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-5a1d1911-fa4d-4e3e-9c99-0a4fca42c580 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:21:31.058576 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:21:31.058576 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:21:31.058576 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:21:31.058576 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:21:31.058576 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:21:31.058576 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:21:31.058576 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:21:31.058576 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:21:31.058576 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:21:31.058576 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:21:31.058576 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:21:31.058576 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:21:31.058576 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:21:31.058576 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:21:31.058576 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:21:31.059768 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:21:31.059768 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:21:31.059768 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:21:31.059768 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:21:31.059768 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:21:31.059768 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:21:31.059768 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:21:31.059768 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:21:31.059768 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:21:31.059768 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:21:31.059768 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:21:31.059768 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:21:31.059768 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:21:31.059768 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:21:31.059768 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:21:31.059768 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:21:31.059768 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:21:31.061057 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:21:31.061057 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:21:31.061057 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:21:31.061057 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 631/1265] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:21:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:21:31.065581 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2127a894-dd25-4b01-a23c-c38637cf3e4b None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:21:31.150128 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:21:31.170614 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2127a894-dd25-4b01-a23c-c38637cf3e4b None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:21:31.203854 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fce1be-c3c1-4481-8902-bd9e8cc88d6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:21:31.204492 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fce1be-c3c1-4481-8902-bd9e8cc88d6b None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:21:31.204895 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fce1be-c3c1-4481-8902-bd9e8cc88d6b None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:21:31.205229 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fce1be-c3c1-4481-8902-bd9e8cc88d6b None admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:21:31.205955 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fce1be-c3c1-4481-8902-bd9e8cc88d6b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:21:31.206420 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fce1be-c3c1-4481-8902-bd9e8cc88d6b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:21:31.207994 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fce1be-c3c1-4481-8902-bd9e8cc88d6b None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:21:31.224214 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 635/1266] 10.222.0.102 () {56 vars in 1114 bytes} [Thu Sep 1 17:21:31 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:21:31.231247 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1d1911-fa4d-4e3e-9c99-0a4fca42c580 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:21:31.332174 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1d1911-fa4d-4e3e-9c99-0a4fca42c580 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:21:31.351536 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:21:31.366071 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7684dab6-b092-433e-b43d-b39c030b4808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:21:31.367205 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7684dab6-b092-433e-b43d-b39c030b4808 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:21:31.367485 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7684dab6-b092-433e-b43d-b39c030b4808 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:21:31.367760 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7684dab6-b092-433e-b43d-b39c030b4808 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:21:31.375569 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7684dab6-b092-433e-b43d-b39c030b4808 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:21:31.377008 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7684dab6-b092-433e-b43d-b39c030b4808 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:21:31.388437 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-7684dab6-b092-433e-b43d-b39c030b4808 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 01 17:21:31.388437 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:21:31.388437 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:21:31.388437 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:21:31.388437 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:21:31.388437 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:21:31.388437 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:21:31.388437 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:21:31.388437 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:21:31.388437 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:21:31.388437 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:21:31.388437 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:21:31.388437 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:21:31.388437 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:21:31.388437 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:21:31.388437 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:21:31.389591 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:21:31.389591 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:21:31.389591 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:21:31.389591 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:21:31.389591 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:21:31.389591 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:21:31.389591 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:21:31.389591 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:21:31.389591 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:21:31.389591 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:21:31.389591 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:21:31.389591 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:21:31.389591 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:21:31.389591 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:21:31.389591 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:21:31.389591 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:21:31.389591 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:21:31.389591 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:21:31.390763 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:21:31.390763 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:21:31.390763 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:21:31.390763 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:21:31.390763 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:21:31.390763 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:21:31.390763 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:21:31.390763 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:21:31.390763 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 01 17:21:31.390763 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:21:31.390763 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 632/1267] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:21:31 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:21:31.395432 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fce1be-c3c1-4481-8902-bd9e8cc88d6b None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:21:31.491703 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fce1be-c3c1-4481-8902-bd9e8cc88d6b None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:21:31.531499 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6db70e-f414-4f57-b14e-bb6c3391dde5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:21:31.532098 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6db70e-f414-4f57-b14e-bb6c3391dde5 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:21:31.532374 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6db70e-f414-4f57-b14e-bb6c3391dde5 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:21:31.532640 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6db70e-f414-4f57-b14e-bb6c3391dde5 None admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:21:31.533373 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6db70e-f414-4f57-b14e-bb6c3391dde5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:21:31.533674 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6db70e-f414-4f57-b14e-bb6c3391dde5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:21:31.535078 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6db70e-f414-4f57-b14e-bb6c3391dde5 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:21:31.544391 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:21:31.552793 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 636/1268] 10.222.0.102 () {56 vars in 1148 bytes} [Thu Sep 1 17:21:31 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 380 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:21:31.559105 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7684dab6-b092-433e-b43d-b39c030b4808 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:21:31.665386 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7684dab6-b092-433e-b43d-b39c030b4808 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:21:31.698674 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073df293-48f7-44e7-8938-0b0ee9706d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:21:31.699306 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073df293-48f7-44e7-8938-0b0ee9706d5f None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:21:31.699589 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073df293-48f7-44e7-8938-0b0ee9706d5f None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:21:31.699863 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073df293-48f7-44e7-8938-0b0ee9706d5f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:21:31.710216 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073df293-48f7-44e7-8938-0b0ee9706d5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:21:31.711669 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073df293-48f7-44e7-8938-0b0ee9706d5f None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:21:31.726191 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-073df293-48f7-44e7-8938-0b0ee9706d5f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:21:31.726191 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:21:31.726191 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:21:31.726191 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:21:31.726191 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:21:31.726191 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:21:31.726191 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:21:31.726191 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:21:31.726191 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:21:31.726191 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:21:31.726191 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:21:31.726191 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:21:31.726191 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:21:31.726191 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:21:31.726191 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:21:31.726191 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:21:31.727119 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:21:31.727119 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:21:31.727119 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:21:31.727119 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:21:31.727119 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:21:31.727119 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:21:31.727119 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:21:31.727119 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:21:31.727119 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:21:31.727119 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:21:31.727119 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:21:31.727119 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:21:31.727119 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:21:31.727119 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:21:31.727119 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:21:31.727119 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:21:31.727119 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:21:31.728048 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:21:31.728048 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:21:31.728048 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:21:31.728048 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 633/1269] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:21:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:21:31.733127 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6db70e-f414-4f57-b14e-bb6c3391dde5 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:21:31.830299 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6db70e-f414-4f57-b14e-bb6c3391dde5 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:21:31.864569 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd72b360-78f4-4478-ae94-609a704235c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:21:31.865168 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd72b360-78f4-4478-ae94-609a704235c1 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:21:31.865488 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd72b360-78f4-4478-ae94-609a704235c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:21:31.865760 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd72b360-78f4-4478-ae94-609a704235c1 None admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:21:31.867192 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd72b360-78f4-4478-ae94-609a704235c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:21:31.867491 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd72b360-78f4-4478-ae94-609a704235c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:21:31.868848 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd72b360-78f4-4478-ae94-609a704235c1 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:21:31.880535 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 637/1270] 10.222.0.102 () {56 vars in 1114 bytes} [Thu Sep 1 17:21:31 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:21:31.887408 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073df293-48f7-44e7-8938-0b0ee9706d5f None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:21:31.920358 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:21:31.989332 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073df293-48f7-44e7-8938-0b0ee9706d5f None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:21:32.022595 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc10a91f-7dca-4d10-b071-03bbc3029b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:21:32.023212 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc10a91f-7dca-4d10-b071-03bbc3029b88 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:21:32.023490 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc10a91f-7dca-4d10-b071-03bbc3029b88 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:21:32.023766 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc10a91f-7dca-4d10-b071-03bbc3029b88 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:21:32.030384 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc10a91f-7dca-4d10-b071-03bbc3029b88 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:21:32.031822 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc10a91f-7dca-4d10-b071-03bbc3029b88 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:21:32.042800 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-bc10a91f-7dca-4d10-b071-03bbc3029b88 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:21:32.042800 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:21:32.042800 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:21:32.042800 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:21:32.042800 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:21:32.042800 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:21:32.042800 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:21:32.042800 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:21:32.042800 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:21:32.042800 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:21:32.042800 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:21:32.042800 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:21:32.042800 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:21:32.042800 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:21:32.042800 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:21:32.043849 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:21:32.043849 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:21:32.043849 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:21:32.043849 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:21:32.043849 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:21:32.043849 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:21:32.043849 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:21:32.043849 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:21:32.043849 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:21:32.043849 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:21:32.043849 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:21:32.043849 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:21:32.043849 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:21:32.043849 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:21:32.043849 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:21:32.043849 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:21:32.043849 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:21:32.045124 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:21:32.045124 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:21:32.045124 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:21:32.045124 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:21:32.045124 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:21:32.045124 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:21:32.045124 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:21:32.045124 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:21:32.045124 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 634/1271] 10.222.0.102 () {56 vars in 1116 bytes} [Thu Sep 1 17:21:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:21:32.049758 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd72b360-78f4-4478-ae94-609a704235c1 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:21:32.106731 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:21:32.148282 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd72b360-78f4-4478-ae94-609a704235c1 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:21:32.181765 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1017bc16-72d7-4df9-8c3a-66ca0869dbc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:21:32.182364 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1017bc16-72d7-4df9-8c3a-66ca0869dbc0 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:21:32.182642 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1017bc16-72d7-4df9-8c3a-66ca0869dbc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:21:32.182922 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1017bc16-72d7-4df9-8c3a-66ca0869dbc0 None admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:21:32.183601 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1017bc16-72d7-4df9-8c3a-66ca0869dbc0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:21:32.183905 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1017bc16-72d7-4df9-8c3a-66ca0869dbc0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:21:32.185339 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1017bc16-72d7-4df9-8c3a-66ca0869dbc0 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:21:32.198039 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 638/1272] 10.222.0.102 () {56 vars in 1153 bytes} [Thu Sep 1 17:21:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:21:32.204074 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc10a91f-7dca-4d10-b071-03bbc3029b88 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:21:32.309139 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:21:32.318340 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc10a91f-7dca-4d10-b071-03bbc3029b88 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:21:32.351192 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a1cad9-80c8-4a91-9556-b7ce6d9479ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:21:32.351841 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a1cad9-80c8-4a91-9556-b7ce6d9479ad None admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:21:32.352118 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a1cad9-80c8-4a91-9556-b7ce6d9479ad None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:21:32.352386 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a1cad9-80c8-4a91-9556-b7ce6d9479ad None admin] PATH_INFO: `/v3/projects/e78fff7f1f6e4f68b8d233ad2e5bbd4f/users/134ef6039cef468bae0e3f3be79b8073/roles/36577a6f46d642d4b81805ede9872490` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:21:32.354295 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a1cad9-80c8-4a91-9556-b7ce6d9479ad None admin] RBAC: Authorizing `identity:create_grant(project_id=e78fff7f1f6e4f68b8d233ad2e5bbd4f, user_id=134ef6039cef468bae0e3f3be79b8073, role_id=36577a6f46d642d4b81805ede9872490)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:21:32.355726 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a1cad9-80c8-4a91-9556-b7ce6d9479ad None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:21:32.368787 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 635/1273] 10.222.0.102 () {58 vars in 1444 bytes} [Thu Sep 1 17:21:32 2022] PUT /identity/v3/projects/e78fff7f1f6e4f68b8d233ad2e5bbd4f/users/134ef6039cef468bae0e3f3be79b8073/roles/36577a6f46d642d4b81805ede9872490 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:21:34.276078 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc98d1ab-2474-47e4-9730-f28fcf37cd8d None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:21:34.276813 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc98d1ab-2474-47e4-9730-f28fcf37cd8d None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:21:34.276813 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc98d1ab-2474-47e4-9730-f28fcf37cd8d None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:21:34.277902 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 639/1274] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:21:34 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:21:34.283652 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d6c7e5-3c64-4336-90d6-d6d7b1b8bf05 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:21:34.284109 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d6c7e5-3c64-4336-90d6-d6d7b1b8bf05 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:21:34.284288 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d6c7e5-3c64-4336-90d6-d6d7b1b8bf05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:21:34.302146 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-87d6c7e5-3c64-4336-90d6-d6d7b1b8bf05 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:21:34.303845 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-87d6c7e5-3c64-4336-90d6-d6d7b1b8bf05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:21:34.369127 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 636/1275] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:21:34 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:21:34.636595 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc98d1ab-2474-47e4-9730-f28fcf37cd8d None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:21:34.737847 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc98d1ab-2474-47e4-9730-f28fcf37cd8d None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:21:34.755279 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:21:34.771822 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4298fcff-4a6e-47bb-a6c7-3f52de526229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:21:34.772468 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4298fcff-4a6e-47bb-a6c7-3f52de526229 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:21:34.772743 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4298fcff-4a6e-47bb-a6c7-3f52de526229 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:21:34.773013 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4298fcff-4a6e-47bb-a6c7-3f52de526229 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:21:34.782690 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4298fcff-4a6e-47bb-a6c7-3f52de526229 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:21:34.787316 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-4298fcff-4a6e-47bb-a6c7-3f52de526229 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:21:34.787316 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:21:34.787316 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:21:34.787316 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:21:34.787316 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:21:34.787316 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:21:34.787316 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:21:34.787316 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:21:34.787316 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:21:34.787316 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:21:34.787316 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:21:34.787316 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:21:34.787316 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:21:34.787316 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:21:34.787316 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:21:34.787316 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:21:34.789043 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:21:34.789043 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:21:34.789043 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:21:34.789043 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:21:34.789043 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:21:34.789043 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:21:34.789043 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:21:34.789043 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:21:34.789043 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:21:34.789043 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:21:34.789043 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:21:34.789043 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:21:34.789043 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:21:34.789043 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:21:34.789043 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:21:34.789043 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:21:34.790002 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:21:34.790002 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:21:34.790002 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:21:34.790002 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:21:34.790002 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:21:34.790002 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:21:34.790002 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:21:34.790002 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:21:34.790002 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:21:34.790002 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:21:34.790002 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:21:34.790002 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:21:34.790002 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 640/1276] 10.222.0.102 () {56 vars in 1101 bytes} [Thu Sep 1 17:21:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:21:34.795004 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d6c7e5-3c64-4336-90d6-d6d7b1b8bf05 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:21:34.875822 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:21:34.895661 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d6c7e5-3c64-4336-90d6-d6d7b1b8bf05 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:21:34.930051 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a48f235-3682-485b-9cd6-b296af765fb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:21:34.930653 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a48f235-3682-485b-9cd6-b296af765fb5 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:21:34.930938 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a48f235-3682-485b-9cd6-b296af765fb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:21:34.931216 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a48f235-3682-485b-9cd6-b296af765fb5 None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:21:34.931880 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a48f235-3682-485b-9cd6-b296af765fb5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:21:34.932188 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a48f235-3682-485b-9cd6-b296af765fb5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:21:34.933591 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a48f235-3682-485b-9cd6-b296af765fb5 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:21:34.941526 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 637/1277] 10.222.0.102 () {56 vars in 1104 bytes} [Thu Sep 1 17:21:34 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:21:34.947308 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4298fcff-4a6e-47bb-a6c7-3f52de526229 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:21:35.045545 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4298fcff-4a6e-47bb-a6c7-3f52de526229 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:21:35.078661 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7b4391-f895-49c8-a230-17788476042a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:21:35.079280 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7b4391-f895-49c8-a230-17788476042a None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:21:35.079557 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7b4391-f895-49c8-a230-17788476042a None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:21:35.079827 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7b4391-f895-49c8-a230-17788476042a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:21:35.095039 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7b4391-f895-49c8-a230-17788476042a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:21:35.096767 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7b4391-f895-49c8-a230-17788476042a None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:21:35.112191 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-4d7b4391-f895-49c8-a230-17788476042a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:21:35.112191 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:21:35.112191 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:21:35.112191 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:21:35.112191 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:21:35.112191 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:21:35.112191 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:21:35.112191 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:21:35.112191 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:21:35.112191 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:21:35.112191 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:21:35.112191 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:21:35.112191 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:21:35.112191 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:21:35.112191 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:21:35.112191 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:21:35.113404 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:21:35.113404 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:21:35.113404 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:21:35.113404 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:21:35.113404 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:21:35.113404 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:21:35.113404 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:21:35.113404 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:21:35.113404 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:21:35.113404 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:21:35.113404 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:21:35.113404 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:21:35.113404 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:21:35.113404 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:21:35.113404 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:21:35.113404 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:21:35.113404 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:21:35.114354 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:21:35.114354 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:21:35.114354 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:21:35.114354 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 641/1278] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:21:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:21:35.119247 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a48f235-3682-485b-9cd6-b296af765fb5 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:21:35.226535 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a48f235-3682-485b-9cd6-b296af765fb5 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:21:35.264296 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656a6262-94b4-44da-a2c9-e0703e9d6457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:21:35.264890 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656a6262-94b4-44da-a2c9-e0703e9d6457 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:21:35.265172 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656a6262-94b4-44da-a2c9-e0703e9d6457 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:21:35.265524 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656a6262-94b4-44da-a2c9-e0703e9d6457 None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:21:35.266172 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656a6262-94b4-44da-a2c9-e0703e9d6457 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:21:35.266479 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656a6262-94b4-44da-a2c9-e0703e9d6457 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:21:35.267995 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656a6262-94b4-44da-a2c9-e0703e9d6457 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:21:35.280575 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 638/1279] 10.222.0.102 () {56 vars in 1114 bytes} [Thu Sep 1 17:21:35 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:21:35.286513 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7b4391-f895-49c8-a230-17788476042a None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:21:35.316780 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:21:35.385361 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7b4391-f895-49c8-a230-17788476042a None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:21:35.417873 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901e4d37-282b-43aa-a346-1abf95853b9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:21:35.418485 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901e4d37-282b-43aa-a346-1abf95853b9a None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:21:35.418796 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901e4d37-282b-43aa-a346-1abf95853b9a None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:21:35.419072 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901e4d37-282b-43aa-a346-1abf95853b9a None admin] PATH_INFO: `/v3/users/placement` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:21:35.426240 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901e4d37-282b-43aa-a346-1abf95853b9a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:21:35.427653 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901e4d37-282b-43aa-a346-1abf95853b9a None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:21:35.438455 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-901e4d37-282b-43aa-a346-1abf95853b9a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 01 17:21:35.438455 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:21:35.438455 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:21:35.438455 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:21:35.438455 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:21:35.438455 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:21:35.438455 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:21:35.438455 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:21:35.438455 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:21:35.438455 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:21:35.438455 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:21:35.438455 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:21:35.438455 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:21:35.438455 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:21:35.438455 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:21:35.438455 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:21:35.439580 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:21:35.439580 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:21:35.439580 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:21:35.439580 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:21:35.439580 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:21:35.439580 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:21:35.439580 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:21:35.439580 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:21:35.439580 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:21:35.439580 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:21:35.439580 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:21:35.439580 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:21:35.439580 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:21:35.439580 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:21:35.439580 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:21:35.439580 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:21:35.439580 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:21:35.439580 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:21:35.440921 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:21:35.440921 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:21:35.440921 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:21:35.440921 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:21:35.440921 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:21:35.440921 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:21:35.440921 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:21:35.440921 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:21:35.440921 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 01 17:21:35.440921 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:21:35.440921 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 642/1280] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:21:35 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:21:35.445972 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656a6262-94b4-44da-a2c9-e0703e9d6457 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:21:35.449834 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:21:35.543983 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656a6262-94b4-44da-a2c9-e0703e9d6457 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:21:35.582228 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3fab98-973c-4a6c-8afd-1634734f0e79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:21:35.582830 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3fab98-973c-4a6c-8afd-1634734f0e79 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:21:35.583113 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3fab98-973c-4a6c-8afd-1634734f0e79 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:21:35.583387 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3fab98-973c-4a6c-8afd-1634734f0e79 None admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:21:35.584109 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3fab98-973c-4a6c-8afd-1634734f0e79 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:21:35.584416 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3fab98-973c-4a6c-8afd-1634734f0e79 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:21:35.586446 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3fab98-973c-4a6c-8afd-1634734f0e79 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:21:35.603536 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 639/1281] 10.222.0.102 () {56 vars in 1148 bytes} [Thu Sep 1 17:21:35 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 380 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:21:35.609573 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901e4d37-282b-43aa-a346-1abf95853b9a None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:21:35.699751 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:21:35.709356 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901e4d37-282b-43aa-a346-1abf95853b9a None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:21:35.741703 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0626374-6127-4bb7-8aff-f09c9b050e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:21:35.742299 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0626374-6127-4bb7-8aff-f09c9b050e7b None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:21:35.742574 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0626374-6127-4bb7-8aff-f09c9b050e7b None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:21:35.742842 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0626374-6127-4bb7-8aff-f09c9b050e7b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:21:35.807109 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0626374-6127-4bb7-8aff-f09c9b050e7b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:21:35.808553 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0626374-6127-4bb7-8aff-f09c9b050e7b None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:21:35.895074 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:21:35.905887 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 643/1282] 10.222.0.102 () {58 vars in 1287 bytes} [Thu Sep 1 17:21:35 2022] GET /identity/v3/auth/tokens => generated 2123 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 17:21:35.911873 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3fab98-973c-4a6c-8afd-1634734f0e79 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:21:35.972869 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:21:36.011714 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3fab98-973c-4a6c-8afd-1634734f0e79 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:21:36.045221 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca958e0-cce1-4913-8358-49f7114bf2ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:21:36.045871 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca958e0-cce1-4913-8358-49f7114bf2ff None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:21:36.046154 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca958e0-cce1-4913-8358-49f7114bf2ff None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:21:36.046428 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca958e0-cce1-4913-8358-49f7114bf2ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:21:36.056210 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca958e0-cce1-4913-8358-49f7114bf2ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:21:36.057744 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca958e0-cce1-4913-8358-49f7114bf2ff None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:21:36.072439 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-bca958e0-cce1-4913-8358-49f7114bf2ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:21:36.072439 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:21:36.072439 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:21:36.072439 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:21:36.072439 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:21:36.072439 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:21:36.072439 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:21:36.072439 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:21:36.072439 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:21:36.072439 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:21:36.072439 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:21:36.072439 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:21:36.072439 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:21:36.072439 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:21:36.072439 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:21:36.072439 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:21:36.073497 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:21:36.073497 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:21:36.073497 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:21:36.073497 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:21:36.073497 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:21:36.073497 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:21:36.073497 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:21:36.073497 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:21:36.073497 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:21:36.073497 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:21:36.073497 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:21:36.073497 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:21:36.073497 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:21:36.073497 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:21:36.073497 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:21:36.073497 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:21:36.073497 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:21:36.074549 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:21:36.074549 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:21:36.074549 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:21:36.074549 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 640/1283] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:21:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:21:36.079509 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0626374-6127-4bb7-8aff-f09c9b050e7b None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:21:36.176303 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0626374-6127-4bb7-8aff-f09c9b050e7b None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:21:36.211146 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93816fb8-1ce3-4121-a475-606d826b9835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:21:36.211860 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93816fb8-1ce3-4121-a475-606d826b9835 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:21:36.212238 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93816fb8-1ce3-4121-a475-606d826b9835 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:21:36.212542 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93816fb8-1ce3-4121-a475-606d826b9835 None admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:21:36.213360 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93816fb8-1ce3-4121-a475-606d826b9835 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:21:36.213705 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93816fb8-1ce3-4121-a475-606d826b9835 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:21:36.215436 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93816fb8-1ce3-4121-a475-606d826b9835 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:21:36.230117 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 644/1284] 10.222.0.102 () {56 vars in 1114 bytes} [Thu Sep 1 17:21:36 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:21:36.236724 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca958e0-cce1-4913-8358-49f7114bf2ff None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:21:36.335656 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca958e0-cce1-4913-8358-49f7114bf2ff None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:21:36.349031 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:21:36.369443 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9ffe6c-d4e1-4854-8e7c-3a8091d5c181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:21:36.370066 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9ffe6c-d4e1-4854-8e7c-3a8091d5c181 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:21:36.370345 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9ffe6c-d4e1-4854-8e7c-3a8091d5c181 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:21:36.370617 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9ffe6c-d4e1-4854-8e7c-3a8091d5c181 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:21:36.377013 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9ffe6c-d4e1-4854-8e7c-3a8091d5c181 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:21:36.378482 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9ffe6c-d4e1-4854-8e7c-3a8091d5c181 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:21:36.390033 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-6a9ffe6c-d4e1-4854-8e7c-3a8091d5c181 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:21:36.390033 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:21:36.390033 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:21:36.390033 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:21:36.390033 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:21:36.390033 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:21:36.390033 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:21:36.390033 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:21:36.390033 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:21:36.390033 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:21:36.390033 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:21:36.390033 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:21:36.390033 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:21:36.390033 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:21:36.390033 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:21:36.391095 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:21:36.391095 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:21:36.391095 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:21:36.391095 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:21:36.391095 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:21:36.391095 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:21:36.391095 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:21:36.391095 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:21:36.391095 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Sep 01 17:21:36.391095 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:21:36.391095 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:21:36.391095 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:21:36.391095 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:21:36.391095 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:21:36.391095 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:21:36.391095 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:21:36.391095 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:21:36.392048 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:21:36.392048 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:21:36.392048 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:21:36.392048 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:21:36.392048 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:21:36.392048 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:21:36.392048 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:21:36.392048 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Sep 01 17:21:36.392048 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 641/1285] 10.222.0.102 () {56 vars in 1116 bytes} [Thu Sep 1 17:21:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:21:36.397029 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93816fb8-1ce3-4121-a475-606d826b9835 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:21:36.446994 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:21:36.497713 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93816fb8-1ce3-4121-a475-606d826b9835 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:21:36.530567 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987f6599-80a9-40ea-b3f7-6eb17e490375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:21:36.531163 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987f6599-80a9-40ea-b3f7-6eb17e490375 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:21:36.531444 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987f6599-80a9-40ea-b3f7-6eb17e490375 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:21:36.531717 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987f6599-80a9-40ea-b3f7-6eb17e490375 None admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:21:36.532397 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987f6599-80a9-40ea-b3f7-6eb17e490375 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:21:36.532704 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987f6599-80a9-40ea-b3f7-6eb17e490375 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:21:36.534163 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987f6599-80a9-40ea-b3f7-6eb17e490375 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:21:36.545842 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 645/1286] 10.222.0.102 () {56 vars in 1153 bytes} [Thu Sep 1 17:21:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:21:36.551918 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9ffe6c-d4e1-4854-8e7c-3a8091d5c181 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:21:36.652177 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9ffe6c-d4e1-4854-8e7c-3a8091d5c181 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:21:36.684891 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d62180-41e0-4983-8184-f9f4158a3c46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:21:36.685506 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d62180-41e0-4983-8184-f9f4158a3c46 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:21:36.685786 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d62180-41e0-4983-8184-f9f4158a3c46 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:21:36.686057 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d62180-41e0-4983-8184-f9f4158a3c46 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:21:36.686767 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d62180-41e0-4983-8184-f9f4158a3c46 None admin] RBAC: Adding query filter params (role.id=36577a6f46d642d4b81805ede9872490, scope.project.id=e78fff7f1f6e4f68b8d233ad2e5bbd4f, user.id=134ef6039cef468bae0e3f3be79b8073) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:21:36.687072 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d62180-41e0-4983-8184-f9f4158a3c46 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:21:36.688503 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d62180-41e0-4983-8184-f9f4158a3c46 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:21:36.694723 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 642/1287] 10.222.0.102 () {56 vars in 1379 bytes} [Thu Sep 1 17:21:36 2022] GET /identity/v3/role_assignments?user.id=134ef6039cef468bae0e3f3be79b8073&scope.project.id=e78fff7f1f6e4f68b8d233ad2e5bbd4f&role.id=36577a6f46d642d4b81805ede9872490 => generated 610 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:21:38.642314 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9b0213-629a-4d65-b4af-868bff6b217d None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:21:38.642913 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9b0213-629a-4d65-b4af-868bff6b217d None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:21:38.642913 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9b0213-629a-4d65-b4af-868bff6b217d None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:21:38.644209 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 646/1288] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:21:38 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:21:38.649827 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c048e79b-e3cc-45d3-bbe2-ffdb093f430b None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:21:38.650230 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c048e79b-e3cc-45d3-bbe2-ffdb093f430b None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:21:38.651075 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c048e79b-e3cc-45d3-bbe2-ffdb093f430b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:21:38.670181 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-c048e79b-e3cc-45d3-bbe2-ffdb093f430b None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:21:38.671358 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-c048e79b-e3cc-45d3-bbe2-ffdb093f430b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:21:38.699583 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:21:38.739985 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 643/1289] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:21:38 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:21:39.004337 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9b0213-629a-4d65-b4af-868bff6b217d None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:21:39.102328 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9b0213-629a-4d65-b4af-868bff6b217d None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:21:39.112534 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:21:39.136370 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d37b3b-475b-42cf-afc2-2d3c7a38e7c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:21:39.136994 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d37b3b-475b-42cf-afc2-2d3c7a38e7c3 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:21:39.137270 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d37b3b-475b-42cf-afc2-2d3c7a38e7c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:21:39.137567 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d37b3b-475b-42cf-afc2-2d3c7a38e7c3 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:21:39.138143 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d37b3b-475b-42cf-afc2-2d3c7a38e7c3 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:21:39.139479 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d37b3b-475b-42cf-afc2-2d3c7a38e7c3 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:21:39.148504 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-b5d37b3b-475b-42cf-afc2-2d3c7a38e7c3 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 17:21:39.148504 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:21:39.148504 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 17:21:39.148504 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 17:21:39.148504 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 17:21:39.148504 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 17:21:39.148504 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 17:21:39.148504 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:21:39.148504 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 17:21:39.148504 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:21:39.148504 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 17:21:39.148504 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:21:39.148504 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:21:39.148504 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:21:39.148504 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:21:39.148504 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:21:39.148504 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:21:39.149727 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:21:39.149727 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:21:39.149727 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:21:39.149727 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:21:39.149727 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:21:39.149727 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:21:39.149727 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 17:21:39.149727 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 17:21:39.149727 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 17:21:39.149727 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 17:21:39.149727 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:21:39.149727 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:21:39.149727 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:21:39.149727 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:21:39.149727 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:21:39.149727 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:21:39.150596 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:21:39.150596 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:21:39.150596 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:21:39.150596 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:21:39.150596 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:21:39.150596 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:21:39.150596 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:21:39.150596 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:21:39.150596 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:21:39.150596 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:21:39.150596 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 17:21:39.150596 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:21:39.150596 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 17:21:39.150596 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:21:39.150596 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 647/1290] 10.222.0.102 () {56 vars in 1122 bytes} [Thu Sep 1 17:21:39 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:21:39.155649 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c048e79b-e3cc-45d3-bbe2-ffdb093f430b None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:21:39.263120 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c048e79b-e3cc-45d3-bbe2-ffdb093f430b None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:21:39.302563 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da92f33-9bde-4a37-9334-006a706d5794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:21:39.303168 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da92f33-9bde-4a37-9334-006a706d5794 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:21:39.303445 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da92f33-9bde-4a37-9334-006a706d5794 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:21:39.303713 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da92f33-9bde-4a37-9334-006a706d5794 None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:21:39.304334 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da92f33-9bde-4a37-9334-006a706d5794 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:21:39.304635 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da92f33-9bde-4a37-9334-006a706d5794 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:21:39.306022 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da92f33-9bde-4a37-9334-006a706d5794 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:21:39.310988 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 644/1291] 10.222.0.102 () {56 vars in 1121 bytes} [Thu Sep 1 17:21:39 2022] GET /identity/v3/services?name=placement => generated 127 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:21:39.316767 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d37b3b-475b-42cf-afc2-2d3c7a38e7c3 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:21:39.413943 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d37b3b-475b-42cf-afc2-2d3c7a38e7c3 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:21:39.446318 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4356d07-c2d4-4c36-b0b5-86f7416ca970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:21:39.447463 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4356d07-c2d4-4c36-b0b5-86f7416ca970 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:21:39.447787 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4356d07-c2d4-4c36-b0b5-86f7416ca970 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:21:39.448049 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4356d07-c2d4-4c36-b0b5-86f7416ca970 None admin] PATH_INFO: `/v3/services` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:21:39.448674 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4356d07-c2d4-4c36-b0b5-86f7416ca970 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:21:39.448972 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4356d07-c2d4-4c36-b0b5-86f7416ca970 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:21:39.450350 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4356d07-c2d4-4c36-b0b5-86f7416ca970 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:21:39.454738 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 648/1292] 10.222.0.102 () {56 vars in 1121 bytes} [Thu Sep 1 17:21:39 2022] GET /identity/v3/services?type=placement => generated 127 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:21:41.351239 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87edacca-bf66-4023-adab-7c3559792441 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:21:41.351803 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87edacca-bf66-4023-adab-7c3559792441 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:21:41.351803 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87edacca-bf66-4023-adab-7c3559792441 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:21:41.353149 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 645/1293] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:21:41 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:21:41.358287 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13073536-970d-4881-ac42-60d255042cf3 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:21:41.358712 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13073536-970d-4881-ac42-60d255042cf3 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:21:41.358844 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13073536-970d-4881-ac42-60d255042cf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:21:41.376600 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-13073536-970d-4881-ac42-60d255042cf3 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:21:41.377841 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-13073536-970d-4881-ac42-60d255042cf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:21:41.406975 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:21:41.443054 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 649/1294] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:21:41 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:21:41.716572 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87edacca-bf66-4023-adab-7c3559792441 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:21:41.748014 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:21:41.818807 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87edacca-bf66-4023-adab-7c3559792441 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:21:41.853057 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f14189a-8d5f-47e1-8d54-be966d0e96c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:21:41.853767 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f14189a-8d5f-47e1-8d54-be966d0e96c5 None admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:21:41.854044 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f14189a-8d5f-47e1-8d54-be966d0e96c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:21:41.854314 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f14189a-8d5f-47e1-8d54-be966d0e96c5 None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:21:41.854960 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f14189a-8d5f-47e1-8d54-be966d0e96c5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:21:41.856321 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f14189a-8d5f-47e1-8d54-be966d0e96c5 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:21:41.870802 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 646/1295] 10.222.0.102 () {60 vars in 1146 bytes} [Thu Sep 1 17:21:41 2022] POST /identity/v3/services => generated 247 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:21:43.799148 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cbaa37-0d12-44ee-a196-578d18bd73af None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:21:43.799678 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cbaa37-0d12-44ee-a196-578d18bd73af None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:21:43.799678 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cbaa37-0d12-44ee-a196-578d18bd73af None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:21:43.801041 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 650/1296] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:21:43 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:21:43.806186 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3921ea73-61ba-498d-9594-c3562e4fd6d8 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:21:43.806733 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3921ea73-61ba-498d-9594-c3562e4fd6d8 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:21:43.806733 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3921ea73-61ba-498d-9594-c3562e4fd6d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:21:43.825258 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-3921ea73-61ba-498d-9594-c3562e4fd6d8 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:21:43.826539 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-3921ea73-61ba-498d-9594-c3562e4fd6d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:21:43.898919 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 647/1297] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:21:43 2022] POST /identity/v3/auth/tokens => generated 2226 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:21:44.168186 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cbaa37-0d12-44ee-a196-578d18bd73af None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:21:44.274053 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cbaa37-0d12-44ee-a196-578d18bd73af None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:21:44.308014 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7b6f9e-2089-4368-bdb2-a47a572e60d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:21:44.308638 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7b6f9e-2089-4368-bdb2-a47a572e60d4 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:21:44.308923 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7b6f9e-2089-4368-bdb2-a47a572e60d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:21:44.309198 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7b6f9e-2089-4368-bdb2-a47a572e60d4 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:21:44.309829 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7b6f9e-2089-4368-bdb2-a47a572e60d4 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:21:44.311169 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7b6f9e-2089-4368-bdb2-a47a572e60d4 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:21:44.319856 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-3d7b6f9e-2089-4368-bdb2-a47a572e60d4 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 17:21:44.319856 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:21:44.319856 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 17:21:44.319856 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 17:21:44.319856 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 17:21:44.319856 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 17:21:44.319856 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 17:21:44.319856 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:21:44.319856 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 17:21:44.319856 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:21:44.319856 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 17:21:44.319856 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:21:44.319856 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:21:44.319856 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:21:44.319856 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:21:44.319856 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:21:44.319856 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:21:44.321028 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:21:44.321028 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:21:44.321028 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:21:44.321028 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:21:44.321028 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:21:44.321028 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:21:44.321028 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 17:21:44.321028 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 17:21:44.321028 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 17:21:44.321028 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 17:21:44.321028 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:21:44.321028 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:21:44.321028 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:21:44.321028 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:21:44.321028 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:21:44.321028 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:21:44.321951 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:21:44.321951 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:21:44.321951 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:21:44.321951 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:21:44.321951 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:21:44.321951 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:21:44.321951 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:21:44.321951 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:21:44.321951 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:21:44.321951 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:21:44.321951 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 17:21:44.321951 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:21:44.321951 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 17:21:44.321951 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:21:44.321951 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 651/1298] 10.222.0.102 () {56 vars in 1122 bytes} [Thu Sep 1 17:21:44 2022] GET /identity/v3/services/placement => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:21:44.327073 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3921ea73-61ba-498d-9594-c3562e4fd6d8 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:21:44.345589 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:21:44.425935 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3921ea73-61ba-498d-9594-c3562e4fd6d8 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:21:44.459168 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ad6cb1-4d61-48b2-b840-e61579011f20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:21:44.459774 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ad6cb1-4d61-48b2-b840-e61579011f20 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:21:44.460055 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ad6cb1-4d61-48b2-b840-e61579011f20 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:21:44.460328 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ad6cb1-4d61-48b2-b840-e61579011f20 None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:21:44.460950 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ad6cb1-4d61-48b2-b840-e61579011f20 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:21:44.461253 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ad6cb1-4d61-48b2-b840-e61579011f20 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:21:44.462654 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ad6cb1-4d61-48b2-b840-e61579011f20 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:21:44.467686 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 648/1299] 10.222.0.102 () {56 vars in 1121 bytes} [Thu Sep 1 17:21:44 2022] GET /identity/v3/services?name=placement => generated 360 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:21:44.473980 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7b6f9e-2089-4368-bdb2-a47a572e60d4 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:21:44.505683 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:21:44.575108 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7b6f9e-2089-4368-bdb2-a47a572e60d4 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:21:44.608296 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0c458a-25f1-45c4-9391-8b158613c79d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:21:44.608894 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0c458a-25f1-45c4-9391-8b158613c79d None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:21:44.609176 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0c458a-25f1-45c4-9391-8b158613c79d None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:21:44.609478 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0c458a-25f1-45c4-9391-8b158613c79d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:21:44.610174 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0c458a-25f1-45c4-9391-8b158613c79d None admin] RBAC: Adding query filter params (interface=public, service_id=ae882cb3ff1d46d195b1ee1c6337b902, region_id=RegionOne) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:21:44.610484 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0c458a-25f1-45c4-9391-8b158613c79d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:21:44.611847 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0c458a-25f1-45c4-9391-8b158613c79d None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:21:44.617288 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 652/1300] 10.222.0.102 () {56 vars in 1256 bytes} [Thu Sep 1 17:21:44 2022] GET /identity/v3/endpoints?service_id=ae882cb3ff1d46d195b1ee1c6337b902&interface=public®ion_id=RegionOne => generated 195 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:21:44.623002 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ad6cb1-4d61-48b2-b840-e61579011f20 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:21:44.704232 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:21:44.723362 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ad6cb1-4d61-48b2-b840-e61579011f20 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:21:44.756985 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e614dc9-be70-409f-bf11-49a7a46bf6fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:21:44.757606 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e614dc9-be70-409f-bf11-49a7a46bf6fe None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:21:44.757883 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e614dc9-be70-409f-bf11-49a7a46bf6fe None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:21:44.758150 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e614dc9-be70-409f-bf11-49a7a46bf6fe None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:21:44.758738 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e614dc9-be70-409f-bf11-49a7a46bf6fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:21:44.760101 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e614dc9-be70-409f-bf11-49a7a46bf6fe None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:21:44.766968 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 649/1301] 10.222.0.102 () {56 vars in 1092 bytes} [Thu Sep 1 17:21:44 2022] GET /identity/v3/services => generated 2201 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:21:46.670984 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633f22a7-e120-4a09-b0c0-93553312d7f6 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:21:46.671518 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633f22a7-e120-4a09-b0c0-93553312d7f6 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:21:46.671518 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633f22a7-e120-4a09-b0c0-93553312d7f6 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:21:46.672878 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 653/1302] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:21:46 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:21:46.678730 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6602ec57-a216-42f7-b55e-6b63de2509b8 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:21:46.679125 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6602ec57-a216-42f7-b55e-6b63de2509b8 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:21:46.679327 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6602ec57-a216-42f7-b55e-6b63de2509b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:21:46.696964 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-6602ec57-a216-42f7-b55e-6b63de2509b8 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:21:46.698167 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-6602ec57-a216-42f7-b55e-6b63de2509b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:21:46.762563 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 650/1303] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:21:46 2022] POST /identity/v3/auth/tokens => generated 2226 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:21:47.031340 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633f22a7-e120-4a09-b0c0-93553312d7f6 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:21:47.123317 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:21:47.131028 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633f22a7-e120-4a09-b0c0-93553312d7f6 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:21:47.163804 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f878d6-3e5f-40b1-b148-0c1958ac60e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:21:47.164422 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f878d6-3e5f-40b1-b148-0c1958ac60e6 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:21:47.164705 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f878d6-3e5f-40b1-b148-0c1958ac60e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:21:47.164976 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f878d6-3e5f-40b1-b148-0c1958ac60e6 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:21:47.165578 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f878d6-3e5f-40b1-b148-0c1958ac60e6 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:21:47.166935 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f878d6-3e5f-40b1-b148-0c1958ac60e6 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:21:47.175451 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-b4f878d6-3e5f-40b1-b148-0c1958ac60e6 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 17:21:47.175451 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:21:47.175451 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 17:21:47.175451 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 17:21:47.175451 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 17:21:47.175451 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 17:21:47.175451 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 17:21:47.175451 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:21:47.175451 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 17:21:47.175451 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:21:47.175451 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 17:21:47.175451 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:21:47.175451 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:21:47.175451 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:21:47.175451 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:21:47.175451 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:21:47.175451 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:21:47.176617 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:21:47.176617 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:21:47.176617 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:21:47.176617 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:21:47.176617 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:21:47.176617 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:21:47.176617 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 17:21:47.176617 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 17:21:47.176617 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 17:21:47.176617 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 17:21:47.176617 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:21:47.176617 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:21:47.176617 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:21:47.176617 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:21:47.176617 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:21:47.176617 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:21:47.178297 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:21:47.178297 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Sep 01 17:21:47.178297 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:21:47.178297 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:21:47.178297 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:21:47.178297 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:21:47.178297 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:21:47.178297 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:21:47.178297 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:21:47.178297 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:21:47.178297 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 17:21:47.178297 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:21:47.178297 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 17:21:47.178297 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Sep 01 17:21:47.178297 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 654/1304] 10.222.0.102 () {56 vars in 1122 bytes} [Thu Sep 1 17:21:47 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:21:47.183461 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6602ec57-a216-42f7-b55e-6b63de2509b8 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:21:47.258261 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:21:47.280417 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6602ec57-a216-42f7-b55e-6b63de2509b8 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:21:47.314289 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d80acc-5d39-4e58-abb2-824b26f4c8b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:21:47.314894 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d80acc-5d39-4e58-abb2-824b26f4c8b4 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:21:47.315169 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d80acc-5d39-4e58-abb2-824b26f4c8b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:21:47.315437 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d80acc-5d39-4e58-abb2-824b26f4c8b4 None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:21:47.316058 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d80acc-5d39-4e58-abb2-824b26f4c8b4 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:21:47.316362 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d80acc-5d39-4e58-abb2-824b26f4c8b4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:21:47.317817 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d80acc-5d39-4e58-abb2-824b26f4c8b4 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:21:47.322866 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 651/1305] 10.222.0.102 () {56 vars in 1121 bytes} [Thu Sep 1 17:21:47 2022] GET /identity/v3/services?name=placement => generated 360 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:21:47.328737 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f878d6-3e5f-40b1-b148-0c1958ac60e6 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:21:47.426622 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f878d6-3e5f-40b1-b148-0c1958ac60e6 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:21:47.459148 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba71af0-3856-439b-9ca2-1481574d879a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:21:47.459745 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba71af0-3856-439b-9ca2-1481574d879a None admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:21:47.460025 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba71af0-3856-439b-9ca2-1481574d879a None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:21:47.460300 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba71af0-3856-439b-9ca2-1481574d879a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:21:47.460969 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba71af0-3856-439b-9ca2-1481574d879a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:21:47.462350 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba71af0-3856-439b-9ca2-1481574d879a None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:21:47.482481 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 655/1306] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:21:47 2022] POST /identity/v3/endpoints => generated 333 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:21:57.319895 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=30 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:21:57.478734 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:51.764298 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18468495-70ec-43bc-970f-1516306b299f None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:51.764907 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18468495-70ec-43bc-970f-1516306b299f None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:51.764907 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18468495-70ec-43bc-970f-1516306b299f None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:51.771719 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236e4440-762b-4233-b2f8-39f4ab60141c None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:51.772184 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236e4440-762b-4233-b2f8-39f4ab60141c None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:51.772266 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236e4440-762b-4233-b2f8-39f4ab60141c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:51.777109 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 652/1307] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:22:51 2022] GET /identity => generated 271 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:22:51.866967 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-236e4440-762b-4233-b2f8-39f4ab60141c None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:22:51.868206 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-236e4440-762b-4233-b2f8-39f4ab60141c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:22:52.005802 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 656/1308] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:22:51 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:22:52.306612 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95c141d-f2e5-48f4-a380-aa40f6a9e4cd None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:52.307632 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95c141d-f2e5-48f4-a380-aa40f6a9e4cd None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:52.308390 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95c141d-f2e5-48f4-a380-aa40f6a9e4cd None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:52.312374 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 653/1309] 10.222.0.102 () {54 vars in 970 bytes} [Thu Sep 1 17:22:52 2022] GET /identity => generated 271 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:22:52.319798 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d98ad6b-ecb0-4603-afc6-5be6c8bb0e74 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:52.320584 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d98ad6b-ecb0-4603-afc6-5be6c8bb0e74 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:52.321587 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d98ad6b-ecb0-4603-afc6-5be6c8bb0e74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:52.430208 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-4d98ad6b-ecb0-4603-afc6-5be6c8bb0e74 None None] MFA Rules not processed for user `7c7d3594fdab4191be1d841af43a5c79`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:22:52.431759 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-4d98ad6b-ecb0-4603-afc6-5be6c8bb0e74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:22:52.466214 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:52.537469 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 657/1310] 10.222.0.102 () {58 vars in 1067 bytes} [Thu Sep 1 17:22:52 2022] POST /identity/v3/auth/tokens => generated 3131 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:22:52.546486 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95c141d-f2e5-48f4-a380-aa40f6a9e4cd None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:52.548326 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-f95c141d-f2e5-48f4-a380-aa40f6a9e4cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:22:52.690175 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95c141d-f2e5-48f4-a380-aa40f6a9e4cd None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:52.796910 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52592cd1-2ef3-41cd-a8b3-82d297769930 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:52.797623 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52592cd1-2ef3-41cd-a8b3-82d297769930 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:52.798039 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52592cd1-2ef3-41cd-a8b3-82d297769930 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:52.798318 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52592cd1-2ef3-41cd-a8b3-82d297769930 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:52.800205 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-52592cd1-2ef3-41cd-a8b3-82d297769930 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:22:52.879544 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:52.953928 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52592cd1-2ef3-41cd-a8b3-82d297769930 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:52.955388 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52592cd1-2ef3-41cd-a8b3-82d297769930 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:52.980892 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 654/1311] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:22:52 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 17:22:55.294617 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d0724a-8a2d-497b-ad1f-bfa97e25f290 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:55.295137 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d0724a-8a2d-497b-ad1f-bfa97e25f290 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:55.295137 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d0724a-8a2d-497b-ad1f-bfa97e25f290 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:55.296449 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 658/1312] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:22:55 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:22:55.301675 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd3b936-e54c-4885-9a91-c244c5ee5343 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:55.302093 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd3b936-e54c-4885-9a91-c244c5ee5343 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:55.302383 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd3b936-e54c-4885-9a91-c244c5ee5343 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:55.379517 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-4fd3b936-e54c-4885-9a91-c244c5ee5343 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:22:55.380771 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-4fd3b936-e54c-4885-9a91-c244c5ee5343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:22:55.487724 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 655/1313] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:22:55 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:22:55.788024 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca44e711-c47f-4dfe-8697-87e5b4ea3f8d None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:55.788489 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca44e711-c47f-4dfe-8697-87e5b4ea3f8d None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:55.788806 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca44e711-c47f-4dfe-8697-87e5b4ea3f8d None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:55.790428 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 659/1314] 10.222.0.102 () {54 vars in 970 bytes} [Thu Sep 1 17:22:55 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:22:55.796800 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80f55f0-fd8a-42c7-ba16-a9682fefd29c None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:55.797374 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80f55f0-fd8a-42c7-ba16-a9682fefd29c None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:55.797529 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80f55f0-fd8a-42c7-ba16-a9682fefd29c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:55.814175 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:55.882759 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-e80f55f0-fd8a-42c7-ba16-a9682fefd29c None None] MFA Rules not processed for user `7c7d3594fdab4191be1d841af43a5c79`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:22:55.883957 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-e80f55f0-fd8a-42c7-ba16-a9682fefd29c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:22:55.961505 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 656/1315] 10.222.0.102 () {58 vars in 1067 bytes} [Thu Sep 1 17:22:55 2022] POST /identity/v3/auth/tokens => generated 3131 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:22:55.970745 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca44e711-c47f-4dfe-8697-87e5b4ea3f8d None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:55.972287 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-ca44e711-c47f-4dfe-8697-87e5b4ea3f8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:22:56.074440 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca44e711-c47f-4dfe-8697-87e5b4ea3f8d None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:56.144221 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:56.147174 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49da13a4-9e0c-4fb8-94e7-bdb3c5801c9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:56.148009 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49da13a4-9e0c-4fb8-94e7-bdb3c5801c9c service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:56.148345 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49da13a4-9e0c-4fb8-94e7-bdb3c5801c9c service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:56.148773 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49da13a4-9e0c-4fb8-94e7-bdb3c5801c9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:56.150624 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-49da13a4-9e0c-4fb8-94e7-bdb3c5801c9c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:22:56.262716 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49da13a4-9e0c-4fb8-94e7-bdb3c5801c9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:56.264186 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49da13a4-9e0c-4fb8-94e7-bdb3c5801c9c service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:56.283879 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 660/1316] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:22:55 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:22:58.720812 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b41ba91-90e9-4f1f-8181-d031e7cb8e3e None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:58.721378 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b41ba91-90e9-4f1f-8181-d031e7cb8e3e None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:58.721378 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b41ba91-90e9-4f1f-8181-d031e7cb8e3e None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:58.722723 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 657/1317] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:22:58 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:22:58.727830 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c87fe7-b18c-4a09-a6cd-781d9f5c266e None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:58.728250 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c87fe7-b18c-4a09-a6cd-781d9f5c266e None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:58.728380 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c87fe7-b18c-4a09-a6cd-781d9f5c266e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:58.793289 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-e6c87fe7-b18c-4a09-a6cd-781d9f5c266e None None] MFA Rules not processed for user `27f309e66b2c4003a820573ada1b0958`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:22:58.794525 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-e6c87fe7-b18c-4a09-a6cd-781d9f5c266e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:22:58.847653 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:58.892986 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 661/1318] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:22:58 2022] POST /identity/v3/auth/tokens => generated 3253 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:22:59.177056 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b41ba91-90e9-4f1f-8181-d031e7cb8e3e None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:59.178650 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-3b41ba91-90e9-4f1f-8181-d031e7cb8e3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:22:59.284232 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b41ba91-90e9-4f1f-8181-d031e7cb8e3e None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:59.287134 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:59.358118 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e24376c-fb7f-47a3-a882-291c4398ea6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:59.358728 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e24376c-fb7f-47a3-a882-291c4398ea6d service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:59.359009 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e24376c-fb7f-47a3-a882-291c4398ea6d service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:59.359282 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e24376c-fb7f-47a3-a882-291c4398ea6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:59.360897 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-7e24376c-fb7f-47a3-a882-291c4398ea6d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:22:59.486424 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e24376c-fb7f-47a3-a882-291c4398ea6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:59.488619 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e24376c-fb7f-47a3-a882-291c4398ea6d service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:59.511569 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 658/1319] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:22:59 2022] GET /identity/v3/auth/tokens => generated 3253 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:23:01.961928 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f85f9bf-a1d4-4f64-8d91-77e0ba33add2 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:01.962493 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f85f9bf-a1d4-4f64-8d91-77e0ba33add2 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:01.962493 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f85f9bf-a1d4-4f64-8d91-77e0ba33add2 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:01.963821 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 662/1320] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:23:01 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Sep 01 17:23:01.970067 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1d918e-bf8e-43ad-912d-97a59f127d30 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:01.970447 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1d918e-bf8e-43ad-912d-97a59f127d30 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:01.970630 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1d918e-bf8e-43ad-912d-97a59f127d30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:02.011379 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:02.037077 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-da1d918e-bf8e-43ad-912d-97a59f127d30 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:23:02.038257 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-da1d918e-bf8e-43ad-912d-97a59f127d30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:23:02.133632 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 659/1321] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:23:01 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:23:02.428562 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f85f9bf-a1d4-4f64-8d91-77e0ba33add2 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:02.430163 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-1f85f9bf-a1d4-4f64-8d91-77e0ba33add2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:23:02.532376 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f85f9bf-a1d4-4f64-8d91-77e0ba33add2 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:02.625414 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e06dddc-5eb6-48c4-ab5a-6e77cf464669 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:02.626020 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e06dddc-5eb6-48c4-ab5a-6e77cf464669 service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:02.626413 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e06dddc-5eb6-48c4-ab5a-6e77cf464669 service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:02.626574 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e06dddc-5eb6-48c4-ab5a-6e77cf464669 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:02.628205 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-2e06dddc-5eb6-48c4-ab5a-6e77cf464669 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:23:02.635664 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:02.737509 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e06dddc-5eb6-48c4-ab5a-6e77cf464669 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:02.738859 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e06dddc-5eb6-48c4-ab5a-6e77cf464669 service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:02.756999 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 663/1322] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:23:02 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:23:05.329057 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65edbf4d-925d-416d-8e6d-a6f97d1c5bbd None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:05.329725 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65edbf4d-925d-416d-8e6d-a6f97d1c5bbd None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:05.329725 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65edbf4d-925d-416d-8e6d-a6f97d1c5bbd None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:05.330981 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 660/1323] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:23:05 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:23:05.336106 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2b0fe1-8d0a-4fcf-be82-ffbf06a916cc None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:05.336533 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2b0fe1-8d0a-4fcf-be82-ffbf06a916cc None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:05.336623 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2b0fe1-8d0a-4fcf-be82-ffbf06a916cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:05.404684 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-ba2b0fe1-8d0a-4fcf-be82-ffbf06a916cc None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:23:05.406004 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-ba2b0fe1-8d0a-4fcf-be82-ffbf06a916cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:23:05.463180 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:05.504023 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 664/1324] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:23:05 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:23:05.792280 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65edbf4d-925d-416d-8e6d-a6f97d1c5bbd None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:05.793913 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-65edbf4d-925d-416d-8e6d-a6f97d1c5bbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:23:05.912430 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65edbf4d-925d-416d-8e6d-a6f97d1c5bbd None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:05.925915 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:05.990305 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a59b79-5a0c-403f-b57b-2cc07b5e6704 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:05.990916 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a59b79-5a0c-403f-b57b-2cc07b5e6704 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:05.991196 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a59b79-5a0c-403f-b57b-2cc07b5e6704 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:05.991467 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a59b79-5a0c-403f-b57b-2cc07b5e6704 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:05.993067 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-82a59b79-5a0c-403f-b57b-2cc07b5e6704 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:23:06.103921 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a59b79-5a0c-403f-b57b-2cc07b5e6704 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:06.105392 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a59b79-5a0c-403f-b57b-2cc07b5e6704 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:06.124499 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 661/1325] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:23:05 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 17:23:08.096598 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b031c119-8a27-4a52-9c74-2b57898b5fc0 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:08.097159 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b031c119-8a27-4a52-9c74-2b57898b5fc0 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:08.097159 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b031c119-8a27-4a52-9c74-2b57898b5fc0 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:08.098483 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 665/1326] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:23:08 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:23:08.103896 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a843a7b2-560d-4dee-ac0b-019735e476ba None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:08.104342 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a843a7b2-560d-4dee-ac0b-019735e476ba None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:08.104454 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a843a7b2-560d-4dee-ac0b-019735e476ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:08.154064 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:08.173646 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-a843a7b2-560d-4dee-ac0b-019735e476ba None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:23:08.174866 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-a843a7b2-560d-4dee-ac0b-019735e476ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:23:08.273617 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 662/1327] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:23:08 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:23:08.560397 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b031c119-8a27-4a52-9c74-2b57898b5fc0 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:08.561989 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-b031c119-8a27-4a52-9c74-2b57898b5fc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:23:08.676166 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b031c119-8a27-4a52-9c74-2b57898b5fc0 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:08.754780 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23857ff9-7ac1-4f6c-83f5-df1c9fc8b757 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:08.755507 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23857ff9-7ac1-4f6c-83f5-df1c9fc8b757 service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:08.755859 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23857ff9-7ac1-4f6c-83f5-df1c9fc8b757 service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:08.756201 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23857ff9-7ac1-4f6c-83f5-df1c9fc8b757 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:08.757939 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-23857ff9-7ac1-4f6c-83f5-df1c9fc8b757 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:23:08.775483 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:08.872128 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23857ff9-7ac1-4f6c-83f5-df1c9fc8b757 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:08.873558 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23857ff9-7ac1-4f6c-83f5-df1c9fc8b757 service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:08.892263 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 666/1328] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:23:08 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:23:10.875632 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded031e8-676e-4e9d-b108-a36572192079 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:10.876145 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded031e8-676e-4e9d-b108-a36572192079 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:10.876241 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded031e8-676e-4e9d-b108-a36572192079 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:10.877607 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 663/1329] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:23:10 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:23:10.882800 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7bc6b1-e455-4ce3-a5b9-268256769efc None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:10.883227 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7bc6b1-e455-4ce3-a5b9-268256769efc None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:10.883328 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7bc6b1-e455-4ce3-a5b9-268256769efc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:10.954918 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-dd7bc6b1-e455-4ce3-a5b9-268256769efc None None] MFA Rules not processed for user `27f309e66b2c4003a820573ada1b0958`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:23:10.956136 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-dd7bc6b1-e455-4ce3-a5b9-268256769efc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:23:11.024169 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:11.057018 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 667/1330] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:23:10 2022] POST /identity/v3/auth/tokens => generated 3253 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:23:11.346151 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded031e8-676e-4e9d-b108-a36572192079 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:11.348207 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-ded031e8-676e-4e9d-b108-a36572192079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:23:11.464631 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded031e8-676e-4e9d-b108-a36572192079 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:11.485753 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:11.540464 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1530a234-c9dd-4e9b-856e-e300b3e9d6d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:11.541077 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1530a234-c9dd-4e9b-856e-e300b3e9d6d9 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:11.541397 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1530a234-c9dd-4e9b-856e-e300b3e9d6d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:11.541675 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1530a234-c9dd-4e9b-856e-e300b3e9d6d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:11.543284 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-1530a234-c9dd-4e9b-856e-e300b3e9d6d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:23:11.654734 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1530a234-c9dd-4e9b-856e-e300b3e9d6d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:11.656106 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1530a234-c9dd-4e9b-856e-e300b3e9d6d9 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:11.674469 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 664/1331] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:23:11 2022] GET /identity/v3/auth/tokens => generated 3253 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:23:14.014025 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b3a563-32ce-4032-92d3-02f976441e9f None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:14.014552 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b3a563-32ce-4032-92d3-02f976441e9f None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:14.014552 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b3a563-32ce-4032-92d3-02f976441e9f None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:14.015924 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 668/1332] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:23:14 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:23:14.025680 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fc0c4f-c933-4faf-9e66-247c38de9443 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:14.026157 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fc0c4f-c933-4faf-9e66-247c38de9443 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:14.026157 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fc0c4f-c933-4faf-9e66-247c38de9443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:14.092742 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:14.095486 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-b4fc0c4f-c933-4faf-9e66-247c38de9443 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:23:14.097210 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-b4fc0c4f-c933-4faf-9e66-247c38de9443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:23:14.191949 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 665/1333] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:23:14 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:23:14.471105 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b3a563-32ce-4032-92d3-02f976441e9f None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:14.472742 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-e9b3a563-32ce-4032-92d3-02f976441e9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:23:14.577436 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b3a563-32ce-4032-92d3-02f976441e9f None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:14.649999 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813230da-8346-4929-98c8-7ae48f391365 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:14.650602 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813230da-8346-4929-98c8-7ae48f391365 service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:14.650876 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813230da-8346-4929-98c8-7ae48f391365 service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:14.651140 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813230da-8346-4929-98c8-7ae48f391365 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:14.652716 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-813230da-8346-4929-98c8-7ae48f391365 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:23:14.683561 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:14.768069 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813230da-8346-4929-98c8-7ae48f391365 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:14.769432 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813230da-8346-4929-98c8-7ae48f391365 service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:14.787714 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 669/1334] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:23:14 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:23:16.738969 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578f4c6c-127a-43bb-ac5d-a04e5c3d1de8 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:16.739537 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578f4c6c-127a-43bb-ac5d-a04e5c3d1de8 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:16.739537 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578f4c6c-127a-43bb-ac5d-a04e5c3d1de8 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:16.740913 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 666/1335] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:23:16 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:23:16.746186 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79198ea1-a6dd-4ca4-818d-50d1b9468f99 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:16.747104 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79198ea1-a6dd-4ca4-818d-50d1b9468f99 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:16.747268 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79198ea1-a6dd-4ca4-818d-50d1b9468f99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:16.812238 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-79198ea1-a6dd-4ca4-818d-50d1b9468f99 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:23:16.813570 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-79198ea1-a6dd-4ca4-818d-50d1b9468f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:23:16.889624 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:16.912063 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 670/1336] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:23:16 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:23:17.196309 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578f4c6c-127a-43bb-ac5d-a04e5c3d1de8 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:17.197902 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-578f4c6c-127a-43bb-ac5d-a04e5c3d1de8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:23:17.302138 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578f4c6c-127a-43bb-ac5d-a04e5c3d1de8 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:17.330276 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:17.379324 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89744e5-c9be-434c-b26a-e41854debae2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:17.379937 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89744e5-c9be-434c-b26a-e41854debae2 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:17.380219 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89744e5-c9be-434c-b26a-e41854debae2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:17.380494 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89744e5-c9be-434c-b26a-e41854debae2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:17.382163 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-b89744e5-c9be-434c-b26a-e41854debae2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:23:17.496090 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89744e5-c9be-434c-b26a-e41854debae2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:17.497515 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89744e5-c9be-434c-b26a-e41854debae2 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:17.515357 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 667/1337] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:23:17 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:23:19.754717 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302394f3-d29f-4fc2-97f6-d44242b0c95a None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:19.755258 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302394f3-d29f-4fc2-97f6-d44242b0c95a None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:19.755258 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302394f3-d29f-4fc2-97f6-d44242b0c95a None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:19.756593 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 671/1338] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:23:19 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:23:19.762481 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803e6518-0dcc-4582-a829-07bf28156ca7 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:19.762949 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803e6518-0dcc-4582-a829-07bf28156ca7 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:19.763029 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803e6518-0dcc-4582-a829-07bf28156ca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:19.833719 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-803e6518-0dcc-4582-a829-07bf28156ca7 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:23:19.834958 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-803e6518-0dcc-4582-a829-07bf28156ca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:23:19.843751 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:19.937768 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 668/1339] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:23:19 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:23:20.224144 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302394f3-d29f-4fc2-97f6-d44242b0c95a None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:20.225782 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-302394f3-d29f-4fc2-97f6-d44242b0c95a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:23:20.329651 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302394f3-d29f-4fc2-97f6-d44242b0c95a None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:20.403239 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2567acc-c9ef-47b0-9be3-bba4f1dfe9aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:20.403852 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2567acc-c9ef-47b0-9be3-bba4f1dfe9aa service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:20.404127 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2567acc-c9ef-47b0-9be3-bba4f1dfe9aa service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:20.404393 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2567acc-c9ef-47b0-9be3-bba4f1dfe9aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:20.406043 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-b2567acc-c9ef-47b0-9be3-bba4f1dfe9aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:23:20.441676 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:20.515495 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2567acc-c9ef-47b0-9be3-bba4f1dfe9aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:20.516820 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2567acc-c9ef-47b0-9be3-bba4f1dfe9aa service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:20.535491 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 672/1340] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:23:20 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:23:24.449769 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4338cb3d-024b-4f3f-ad3d-73646ca4aafa None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:24.450299 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4338cb3d-024b-4f3f-ad3d-73646ca4aafa None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:24.450299 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4338cb3d-024b-4f3f-ad3d-73646ca4aafa None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:24.451885 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 669/1341] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:23:24 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:23:24.456935 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db8784a-f5fe-4d12-8c71-5ad26ab2d9c7 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:24.457372 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db8784a-f5fe-4d12-8c71-5ad26ab2d9c7 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:24.457538 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db8784a-f5fe-4d12-8c71-5ad26ab2d9c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:24.525789 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-4db8784a-f5fe-4d12-8c71-5ad26ab2d9c7 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:23:24.527550 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-4db8784a-f5fe-4d12-8c71-5ad26ab2d9c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:23:24.617511 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:24.629356 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 673/1342] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:23:24 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:23:24.913616 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4338cb3d-024b-4f3f-ad3d-73646ca4aafa None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:24.915230 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-4338cb3d-024b-4f3f-ad3d-73646ca4aafa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:23:25.023395 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4338cb3d-024b-4f3f-ad3d-73646ca4aafa None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:25.064951 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:25.103675 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ee9bf8-2e43-4896-a82d-8deebc2d017b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:25.104320 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ee9bf8-2e43-4896-a82d-8deebc2d017b service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:25.104606 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ee9bf8-2e43-4896-a82d-8deebc2d017b service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:25.104881 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ee9bf8-2e43-4896-a82d-8deebc2d017b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:25.107214 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-05ee9bf8-2e43-4896-a82d-8deebc2d017b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:23:25.231117 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ee9bf8-2e43-4896-a82d-8deebc2d017b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:25.232602 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ee9bf8-2e43-4896-a82d-8deebc2d017b service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:25.250818 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 670/1343] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:23:24 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:23:27.941596 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d93bcd-0eda-46fb-b2f5-df57234f968e None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:27.942131 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d93bcd-0eda-46fb-b2f5-df57234f968e None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:27.942131 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d93bcd-0eda-46fb-b2f5-df57234f968e None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:27.943478 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 674/1344] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:23:27 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:23:27.948767 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da76ebf6-425e-4a85-a34f-8698f563b19e None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:27.949177 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da76ebf6-425e-4a85-a34f-8698f563b19e None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:27.949339 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da76ebf6-425e-4a85-a34f-8698f563b19e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:28.016954 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-da76ebf6-425e-4a85-a34f-8698f563b19e None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:23:28.018826 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-da76ebf6-425e-4a85-a34f-8698f563b19e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:23:28.040343 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:28.123713 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 671/1345] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:23:27 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:23:28.406287 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d93bcd-0eda-46fb-b2f5-df57234f968e None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:28.407838 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-e5d93bcd-0eda-46fb-b2f5-df57234f968e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:23:28.520013 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d93bcd-0eda-46fb-b2f5-df57234f968e None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:28.600165 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee1d68d-d80e-4647-b787-585c50a25dc6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:28.600771 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee1d68d-d80e-4647-b787-585c50a25dc6 service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:28.601047 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee1d68d-d80e-4647-b787-585c50a25dc6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:28.601333 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee1d68d-d80e-4647-b787-585c50a25dc6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:28.602924 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-0ee1d68d-d80e-4647-b787-585c50a25dc6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:23:28.648174 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:28.721106 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee1d68d-d80e-4647-b787-585c50a25dc6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:28.722622 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee1d68d-d80e-4647-b787-585c50a25dc6 service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:28.742940 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 675/1346] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:23:28 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 17:23:32.687328 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ab5f06-64db-4372-9cf3-1c39f61627fd None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:32.687775 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ab5f06-64db-4372-9cf3-1c39f61627fd None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:32.688028 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ab5f06-64db-4372-9cf3-1c39f61627fd None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:32.689516 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 672/1347] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:23:32 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:23:32.695290 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903ae8de-f893-49f1-9fe0-4d4b2f17d8c5 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:32.695756 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903ae8de-f893-49f1-9fe0-4d4b2f17d8c5 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:32.695862 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903ae8de-f893-49f1-9fe0-4d4b2f17d8c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:32.797575 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-903ae8de-f893-49f1-9fe0-4d4b2f17d8c5 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:23:32.798856 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-903ae8de-f893-49f1-9fe0-4d4b2f17d8c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:23:32.902117 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:32.905260 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 676/1348] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:23:32 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:23:33.196536 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ab5f06-64db-4372-9cf3-1c39f61627fd None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:33.198181 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-a7ab5f06-64db-4372-9cf3-1c39f61627fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:23:33.311541 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ab5f06-64db-4372-9cf3-1c39f61627fd None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:33.362687 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:33.390965 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa5b998-5a34-4acb-9250-dc9b7fa3d725 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:33.391503 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa5b998-5a34-4acb-9250-dc9b7fa3d725 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:33.391780 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa5b998-5a34-4acb-9250-dc9b7fa3d725 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:33.392117 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa5b998-5a34-4acb-9250-dc9b7fa3d725 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:33.393781 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-8fa5b998-5a34-4acb-9250-dc9b7fa3d725 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:23:33.507340 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa5b998-5a34-4acb-9250-dc9b7fa3d725 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:33.508793 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa5b998-5a34-4acb-9250-dc9b7fa3d725 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:33.528064 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 673/1349] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:23:33 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:23:38.974757 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9d81d1-a8fd-4abd-a953-d48441dc7cc3 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:38.977864 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9d81d1-a8fd-4abd-a953-d48441dc7cc3 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:38.978341 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9d81d1-a8fd-4abd-a953-d48441dc7cc3 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:38.980725 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 677/1350] 10.222.0.102 () {54 vars in 926 bytes} [Thu Sep 1 17:23:38 2022] GET /identity => generated 271 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:23:38.989403 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0375b0-78e2-4125-b661-5198196e36b5 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:39.000051 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0375b0-78e2-4125-b661-5198196e36b5 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:39.000051 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0375b0-78e2-4125-b661-5198196e36b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:39.086874 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-7a0375b0-78e2-4125-b661-5198196e36b5 None None] MFA Rules not processed for user `134ef6039cef468bae0e3f3be79b8073`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:23:39.089287 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-7a0375b0-78e2-4125-b661-5198196e36b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:23:39.111029 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:39.208771 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 674/1351] 10.222.0.102 () {58 vars in 1023 bytes} [Thu Sep 1 17:23:38 2022] POST /identity/v3/auth/tokens => generated 3318 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:23:43.194838 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3378cd-0c0b-407e-a2c9-16cc525a17b3 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:43.195328 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3378cd-0c0b-407e-a2c9-16cc525a17b3 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:43.195328 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3378cd-0c0b-407e-a2c9-16cc525a17b3 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:43.197335 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 678/1352] 10.222.0.102 () {54 vars in 926 bytes} [Thu Sep 1 17:23:43 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:23:43.202844 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f7dd9e-5791-4386-984a-bcb9cb90c04c None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:43.203297 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f7dd9e-5791-4386-984a-bcb9cb90c04c None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:43.203417 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f7dd9e-5791-4386-984a-bcb9cb90c04c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:43.287811 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-83f7dd9e-5791-4386-984a-bcb9cb90c04c None None] MFA Rules not processed for user `134ef6039cef468bae0e3f3be79b8073`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:23:43.289040 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-83f7dd9e-5791-4386-984a-bcb9cb90c04c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:23:43.365931 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:43.404848 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 675/1353] 10.222.0.102 () {58 vars in 1023 bytes} [Thu Sep 1 17:23:43 2022] POST /identity/v3/auth/tokens => generated 3318 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:23:44.843728 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e0208d-beaf-426a-ae74-71dd22f1a523 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:44.844263 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e0208d-beaf-426a-ae74-71dd22f1a523 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:44.844263 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e0208d-beaf-426a-ae74-71dd22f1a523 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:44.845657 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 679/1354] 10.222.0.102 () {54 vars in 926 bytes} [Thu Sep 1 17:23:44 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:23:44.850818 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c590c22c-1cc6-4db3-8e90-2f578dd40ab2 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:44.851898 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c590c22c-1cc6-4db3-8e90-2f578dd40ab2 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:44.851898 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c590c22c-1cc6-4db3-8e90-2f578dd40ab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:44.931595 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-c590c22c-1cc6-4db3-8e90-2f578dd40ab2 None None] MFA Rules not processed for user `134ef6039cef468bae0e3f3be79b8073`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:23:44.932822 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-c590c22c-1cc6-4db3-8e90-2f578dd40ab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:23:45.041519 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 676/1355] 10.222.0.102 () {58 vars in 1023 bytes} [Thu Sep 1 17:23:44 2022] POST /identity/v3/auth/tokens => generated 3318 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:23:48.484378 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14861105-371a-454f-8550-d8c2f45b8740 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:48.484910 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14861105-371a-454f-8550-d8c2f45b8740 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:48.485101 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14861105-371a-454f-8550-d8c2f45b8740 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:48.486544 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 680/1356] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:23:48 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:23:48.492125 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af31dc0d-e91e-44fd-83cc-177c7500a857 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:48.492503 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af31dc0d-e91e-44fd-83cc-177c7500a857 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:48.492836 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af31dc0d-e91e-44fd-83cc-177c7500a857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:48.540527 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:48.588166 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-af31dc0d-e91e-44fd-83cc-177c7500a857 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:23:48.589671 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-af31dc0d-e91e-44fd-83cc-177c7500a857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:23:48.713052 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 677/1357] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:23:48 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:23:49.014232 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c93696-fc9f-4959-8476-51e11e9b3faa None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:49.014513 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c93696-fc9f-4959-8476-51e11e9b3faa None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:49.015014 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c93696-fc9f-4959-8476-51e11e9b3faa None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:49.016500 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 681/1358] 10.222.0.102 () {54 vars in 967 bytes} [Thu Sep 1 17:23:49 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Sep 01 17:23:49.026100 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c821895-d0fc-482d-aa0b-94f22846deff None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:49.026830 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c821895-d0fc-482d-aa0b-94f22846deff None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:49.028189 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c821895-d0fc-482d-aa0b-94f22846deff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:49.122398 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-9c821895-d0fc-482d-aa0b-94f22846deff None None] MFA Rules not processed for user `5f8293316baf4774b62fc11d55d50865`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:23:49.123610 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-9c821895-d0fc-482d-aa0b-94f22846deff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:23:49.177604 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:49.254847 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 678/1359] 10.222.0.102 () {58 vars in 1064 bytes} [Thu Sep 1 17:23:49 2022] POST /identity/v3/auth/tokens => generated 3382 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:23:49.262549 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c93696-fc9f-4959-8476-51e11e9b3faa None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:49.264091 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-67c93696-fc9f-4959-8476-51e11e9b3faa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:23:49.418455 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c93696-fc9f-4959-8476-51e11e9b3faa None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:49.499667 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:49.542419 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1e8d98-2465-4ca6-a3d0-d19963742211 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f8293316baf4774b62fc11d55d50865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:49.542943 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1e8d98-2465-4ca6-a3d0-d19963742211 service nova] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:49.543220 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1e8d98-2465-4ca6-a3d0-d19963742211 service nova] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:49.543666 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1e8d98-2465-4ca6-a3d0-d19963742211 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:49.545333 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-9e1e8d98-2465-4ca6-a3d0-d19963742211 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:23:49.640225 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1e8d98-2465-4ca6-a3d0-d19963742211 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:49.642084 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1e8d98-2465-4ca6-a3d0-d19963742211 service nova] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:49.660980 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 682/1360] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:23:49 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:23:51.878079 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acc8e40-bc9f-4390-9b0f-51fe3bd1413b None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:51.878608 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acc8e40-bc9f-4390-9b0f-51fe3bd1413b None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:51.878608 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acc8e40-bc9f-4390-9b0f-51fe3bd1413b None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:51.880240 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 679/1361] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:23:51 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:23:51.886204 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8965d2d4-085b-460c-b223-4a4077196134 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:51.886628 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8965d2d4-085b-460c-b223-4a4077196134 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:51.886790 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8965d2d4-085b-460c-b223-4a4077196134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:51.959382 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-8965d2d4-085b-460c-b223-4a4077196134 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:23:51.960719 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-8965d2d4-085b-460c-b223-4a4077196134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:23:52.002196 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:52.077690 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 683/1362] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:23:51 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:23:52.368764 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acc8e40-bc9f-4390-9b0f-51fe3bd1413b None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:52.372497 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-3acc8e40-bc9f-4390-9b0f-51fe3bd1413b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:23:52.549901 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acc8e40-bc9f-4390-9b0f-51fe3bd1413b None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:52.564078 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:52.687478 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19ed08d-37f8-4b06-8af9-4d4069baf324 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f8293316baf4774b62fc11d55d50865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:52.688139 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ed08d-37f8-4b06-8af9-4d4069baf324 service nova] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:52.688497 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ed08d-37f8-4b06-8af9-4d4069baf324 service nova] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:52.688880 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ed08d-37f8-4b06-8af9-4d4069baf324 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:52.691441 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-e19ed08d-37f8-4b06-8af9-4d4069baf324 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:23:52.800538 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19ed08d-37f8-4b06-8af9-4d4069baf324 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:52.802336 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19ed08d-37f8-4b06-8af9-4d4069baf324 service nova] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:52.825783 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 680/1363] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:23:52 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:23:55.089577 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cede1ed0-b2b0-43cb-9a73-8bc556c3d9f7 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:55.090440 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cede1ed0-b2b0-43cb-9a73-8bc556c3d9f7 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:55.090879 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cede1ed0-b2b0-43cb-9a73-8bc556c3d9f7 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:55.092402 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 684/1364] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:23:55 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:23:55.098103 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5020c36c-0608-49fe-b170-251e6f213a7d None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:55.098618 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5020c36c-0608-49fe-b170-251e6f213a7d None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:55.098724 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5020c36c-0608-49fe-b170-251e6f213a7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:55.117536 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:55.173877 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-5020c36c-0608-49fe-b170-251e6f213a7d None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:23:55.175089 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-5020c36c-0608-49fe-b170-251e6f213a7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:23:55.292457 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 681/1365] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:23:55 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:23:55.583169 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d33f106-7837-479e-a264-6e43a779ddec None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:55.583454 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d33f106-7837-479e-a264-6e43a779ddec None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:55.583733 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d33f106-7837-479e-a264-6e43a779ddec None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:55.585237 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 685/1366] 10.222.0.102 () {54 vars in 967 bytes} [Thu Sep 1 17:23:55 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:23:55.591367 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccd3f21-fbd9-4b14-bc83-3dd4573a6356 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:55.591654 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccd3f21-fbd9-4b14-bc83-3dd4573a6356 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:55.591942 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccd3f21-fbd9-4b14-bc83-3dd4573a6356 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:55.672100 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-bccd3f21-fbd9-4b14-bc83-3dd4573a6356 None None] MFA Rules not processed for user `5f8293316baf4774b62fc11d55d50865`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:23:55.673213 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-bccd3f21-fbd9-4b14-bc83-3dd4573a6356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:23:55.712158 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:55.801995 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 682/1367] 10.222.0.102 () {58 vars in 1064 bytes} [Thu Sep 1 17:23:55 2022] POST /identity/v3/auth/tokens => generated 3382 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:23:55.814489 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d33f106-7837-479e-a264-6e43a779ddec None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:55.816271 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-6d33f106-7837-479e-a264-6e43a779ddec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:23:55.980793 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d33f106-7837-479e-a264-6e43a779ddec None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:56.004450 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:56.104419 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc175b7b-3b0f-448c-8cac-243d90987cf4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f8293316baf4774b62fc11d55d50865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:56.105702 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc175b7b-3b0f-448c-8cac-243d90987cf4 service nova] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:56.105702 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc175b7b-3b0f-448c-8cac-243d90987cf4 service nova] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:56.105702 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc175b7b-3b0f-448c-8cac-243d90987cf4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:56.107245 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-bc175b7b-3b0f-448c-8cac-243d90987cf4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:23:56.214039 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc175b7b-3b0f-448c-8cac-243d90987cf4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:56.216206 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc175b7b-3b0f-448c-8cac-243d90987cf4 service nova] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:56.238708 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 686/1368] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:23:55 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:23:56.811515 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db97b53-629a-4a92-b8e4-6f94865a68db None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:56.812061 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db97b53-629a-4a92-b8e4-6f94865a68db None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:56.812061 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db97b53-629a-4a92-b8e4-6f94865a68db None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:56.816244 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 683/1369] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:23:56 2022] GET /identity => generated 271 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:23:56.818960 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e262de-474b-4bff-a8f3-378f15557f53 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:56.819252 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e262de-474b-4bff-a8f3-378f15557f53 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:56.819594 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e262de-474b-4bff-a8f3-378f15557f53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:56.857086 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:56.897127 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-15e262de-474b-4bff-a8f3-378f15557f53 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:23:56.898578 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-15e262de-474b-4bff-a8f3-378f15557f53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:23:57.005345 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 687/1370] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:23:56 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:23:57.604325 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc470f82-3f51-428b-bbea-517bb6195d72 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:57.604612 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc470f82-3f51-428b-bbea-517bb6195d72 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:57.604891 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc470f82-3f51-428b-bbea-517bb6195d72 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:57.606439 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 684/1371] 10.222.0.102 () {54 vars in 969 bytes} [Thu Sep 1 17:23:57 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:23:57.611235 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bdcd6b-c069-4a1b-8ea1-f18704aeafec None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:57.611693 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bdcd6b-c069-4a1b-8ea1-f18704aeafec None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:57.611786 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bdcd6b-c069-4a1b-8ea1-f18704aeafec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:57.694937 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-20bdcd6b-c069-4a1b-8ea1-f18704aeafec None None] MFA Rules not processed for user `523ed589872a45a09b6a7675a7fb0b75`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:23:57.696229 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-20bdcd6b-c069-4a1b-8ea1-f18704aeafec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:23:57.748891 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:57.782076 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 688/1372] 10.222.0.102 () {58 vars in 1066 bytes} [Thu Sep 1 17:23:57 2022] POST /identity/v3/auth/tokens => generated 3130 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:23:57.790787 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc470f82-3f51-428b-bbea-517bb6195d72 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:57.792425 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-fc470f82-3f51-428b-bbea-517bb6195d72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:23:57.911112 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc470f82-3f51-428b-bbea-517bb6195d72 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:57.964386 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:58.005822 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadc0a45-0089-4631-8cfb-a4911ed4eb1e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '523ed589872a45a09b6a7675a7fb0b75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:58.006432 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadc0a45-0089-4631-8cfb-a4911ed4eb1e service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:58.006743 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadc0a45-0089-4631-8cfb-a4911ed4eb1e service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:58.007042 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadc0a45-0089-4631-8cfb-a4911ed4eb1e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:58.009010 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-aadc0a45-0089-4631-8cfb-a4911ed4eb1e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:23:58.208456 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadc0a45-0089-4631-8cfb-a4911ed4eb1e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:58.209875 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadc0a45-0089-4631-8cfb-a4911ed4eb1e service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:58.232539 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 685/1373] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:23:57 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:23:58.263881 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7932d20b-4a91-462d-b6fe-0ae498c918ff None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:58.264229 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7932d20b-4a91-462d-b6fe-0ae498c918ff None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:58.264587 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7932d20b-4a91-462d-b6fe-0ae498c918ff None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:58.266204 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 689/1374] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:23:58 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:23:58.271486 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9776889e-830a-4838-a1b4-a4133f73e86d None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:58.271701 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9776889e-830a-4838-a1b4-a4133f73e86d None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:58.271982 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9776889e-830a-4838-a1b4-a4133f73e86d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:58.357953 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-9776889e-830a-4838-a1b4-a4133f73e86d None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:23:58.359056 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-9776889e-830a-4838-a1b4-a4133f73e86d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:23:58.382882 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:58.472685 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 686/1375] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:23:58 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:23:58.762788 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7932d20b-4a91-462d-b6fe-0ae498c918ff None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:58.764428 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-7932d20b-4a91-462d-b6fe-0ae498c918ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:23:58.922277 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7932d20b-4a91-462d-b6fe-0ae498c918ff None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:58.981789 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:59.043535 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb2f57d-2fdf-49f2-b881-20cb2ab2f9cc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f8293316baf4774b62fc11d55d50865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:59.044060 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb2f57d-2fdf-49f2-b881-20cb2ab2f9cc service nova] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:59.044336 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb2f57d-2fdf-49f2-b881-20cb2ab2f9cc service nova] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:59.044604 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb2f57d-2fdf-49f2-b881-20cb2ab2f9cc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:59.046228 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-ecb2f57d-2fdf-49f2-b881-20cb2ab2f9cc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:23:59.139387 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb2f57d-2fdf-49f2-b881-20cb2ab2f9cc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:59.141016 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb2f57d-2fdf-49f2-b881-20cb2ab2f9cc service nova] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:59.158825 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 690/1376] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:23:58 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 17:24:01.142671 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f13dc2c-cbfb-4950-b487-f40730e8d664 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:01.142671 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f13dc2c-cbfb-4950-b487-f40730e8d664 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:01.143251 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f13dc2c-cbfb-4950-b487-f40730e8d664 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:01.144304 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 687/1377] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:24:01 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:24:01.150169 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2a0a4c-a75f-4271-b5a3-4298924e71e0 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:01.150599 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2a0a4c-a75f-4271-b5a3-4298924e71e0 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:01.150710 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2a0a4c-a75f-4271-b5a3-4298924e71e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:01.217260 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-bb2a0a4c-a75f-4271-b5a3-4298924e71e0 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:24:01.218495 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-bb2a0a4c-a75f-4271-b5a3-4298924e71e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:01.225492 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:01.343449 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 691/1378] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:24:01 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:24:01.633180 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f13dc2c-cbfb-4950-b487-f40730e8d664 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:01.634721 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-4f13dc2c-cbfb-4950-b487-f40730e8d664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:01.814019 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f13dc2c-cbfb-4950-b487-f40730e8d664 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:01.817199 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:01.987403 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4234f97f-d8e4-4560-a8a9-8944a11f9ecb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f8293316baf4774b62fc11d55d50865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:01.988382 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4234f97f-d8e4-4560-a8a9-8944a11f9ecb service nova] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:01.988918 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4234f97f-d8e4-4560-a8a9-8944a11f9ecb service nova] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:01.989458 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4234f97f-d8e4-4560-a8a9-8944a11f9ecb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:01.992012 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-4234f97f-d8e4-4560-a8a9-8944a11f9ecb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:02.113001 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4234f97f-d8e4-4560-a8a9-8944a11f9ecb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:02.114636 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4234f97f-d8e4-4560-a8a9-8944a11f9ecb service nova] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:02.140395 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 688/1379] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:24:01 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 507 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:24:04.106062 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bedc40a-7c77-400f-a47a-98bfb2fdb18c None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:04.106590 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bedc40a-7c77-400f-a47a-98bfb2fdb18c None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:04.106590 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bedc40a-7c77-400f-a47a-98bfb2fdb18c None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:04.108042 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 692/1380] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:24:04 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:24:04.113067 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a365cfe0-2ec8-416e-90cb-364074dc8439 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:04.113511 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a365cfe0-2ec8-416e-90cb-364074dc8439 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:04.113696 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a365cfe0-2ec8-416e-90cb-364074dc8439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:04.125735 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:04.191834 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-a365cfe0-2ec8-416e-90cb-364074dc8439 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:24:04.193041 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-a365cfe0-2ec8-416e-90cb-364074dc8439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:04.317146 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 689/1381] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:24:04 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:24:04.609364 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bedc40a-7c77-400f-a47a-98bfb2fdb18c None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:04.610946 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-8bedc40a-7c77-400f-a47a-98bfb2fdb18c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:04.916896 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:04.935080 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bedc40a-7c77-400f-a47a-98bfb2fdb18c None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:05.068390 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042ab6cd-ca6c-4b4f-9dae-f2f6feaf5acf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f8293316baf4774b62fc11d55d50865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:05.069169 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042ab6cd-ca6c-4b4f-9dae-f2f6feaf5acf service nova] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:05.070117 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042ab6cd-ca6c-4b4f-9dae-f2f6feaf5acf service nova] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:05.070492 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042ab6cd-ca6c-4b4f-9dae-f2f6feaf5acf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:05.072191 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-042ab6cd-ca6c-4b4f-9dae-f2f6feaf5acf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:05.168734 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042ab6cd-ca6c-4b4f-9dae-f2f6feaf5acf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:05.170297 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042ab6cd-ca6c-4b4f-9dae-f2f6feaf5acf service nova] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:05.180969 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:05.191229 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 693/1382] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:24:04 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 582 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:24:07.183373 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590a3d95-dc29-4ae4-a842-dc4eec34d8c6 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:07.183935 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590a3d95-dc29-4ae4-a842-dc4eec34d8c6 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:07.184033 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590a3d95-dc29-4ae4-a842-dc4eec34d8c6 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:07.185814 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 690/1383] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:24:07 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:24:07.191632 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d7f0fc-4fce-4555-a124-1103bf4ccf83 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:07.192162 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d7f0fc-4fce-4555-a124-1103bf4ccf83 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:07.192281 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d7f0fc-4fce-4555-a124-1103bf4ccf83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:07.273356 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-b0d7f0fc-4fce-4555-a124-1103bf4ccf83 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:24:07.274623 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-b0d7f0fc-4fce-4555-a124-1103bf4ccf83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:07.380056 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 694/1384] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:24:07 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:24:07.665737 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590a3d95-dc29-4ae4-a842-dc4eec34d8c6 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:07.667318 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-590a3d95-dc29-4ae4-a842-dc4eec34d8c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:07.759121 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:07.828899 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590a3d95-dc29-4ae4-a842-dc4eec34d8c6 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:07.960614 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7208a971-4ffd-47e2-8680-c5d87ef2a712 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f8293316baf4774b62fc11d55d50865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:07.961433 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7208a971-4ffd-47e2-8680-c5d87ef2a712 service nova] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:07.961808 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7208a971-4ffd-47e2-8680-c5d87ef2a712 service nova] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:07.962180 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7208a971-4ffd-47e2-8680-c5d87ef2a712 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:07.964274 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-7208a971-4ffd-47e2-8680-c5d87ef2a712 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:08.040792 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:08.089552 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7208a971-4ffd-47e2-8680-c5d87ef2a712 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:08.091288 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7208a971-4ffd-47e2-8680-c5d87ef2a712 service nova] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:08.112469 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 691/1385] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:24:07 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:24:10.259524 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267d1e7f-f3b3-4cf0-b565-aeac1e1c7e5e None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:10.260108 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267d1e7f-f3b3-4cf0-b565-aeac1e1c7e5e None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:10.260108 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267d1e7f-f3b3-4cf0-b565-aeac1e1c7e5e None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:10.261597 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 695/1386] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:24:10 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:24:10.267970 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67ea783-46f4-4c88-9b3a-34b1493df45d None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:10.268444 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67ea783-46f4-4c88-9b3a-34b1493df45d None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:10.268635 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67ea783-46f4-4c88-9b3a-34b1493df45d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:10.341083 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-f67ea783-46f4-4c88-9b3a-34b1493df45d None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:24:10.342470 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-f67ea783-46f4-4c88-9b3a-34b1493df45d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:10.456177 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 692/1387] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:24:10 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:24:10.767371 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267d1e7f-f3b3-4cf0-b565-aeac1e1c7e5e None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:10.769175 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-267d1e7f-f3b3-4cf0-b565-aeac1e1c7e5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:10.884549 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:10.977610 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267d1e7f-f3b3-4cf0-b565-aeac1e1c7e5e None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:11.138926 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24bf7ba-d888-429c-8144-17a0c4d07382 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f8293316baf4774b62fc11d55d50865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:11.139686 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24bf7ba-d888-429c-8144-17a0c4d07382 service nova] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:11.140043 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24bf7ba-d888-429c-8144-17a0c4d07382 service nova] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:11.140401 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24bf7ba-d888-429c-8144-17a0c4d07382 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:11.142573 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-f24bf7ba-d888-429c-8144-17a0c4d07382 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:11.192152 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:11.250893 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24bf7ba-d888-429c-8144-17a0c4d07382 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:11.252506 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24bf7ba-d888-429c-8144-17a0c4d07382 service nova] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:11.271574 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 696/1388] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:24:10 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 505 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:24:13.263804 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dd62dd-5d2e-48d2-8423-27b40da0da83 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:13.264339 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dd62dd-5d2e-48d2-8423-27b40da0da83 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:13.264339 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dd62dd-5d2e-48d2-8423-27b40da0da83 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:13.265699 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 693/1389] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:24:13 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Sep 01 17:24:13.271214 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ff369c-e666-416c-9399-5dac51c0efe5 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:13.271686 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ff369c-e666-416c-9399-5dac51c0efe5 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:13.271793 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ff369c-e666-416c-9399-5dac51c0efe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:13.341998 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-81ff369c-e666-416c-9399-5dac51c0efe5 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:24:13.343204 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-81ff369c-e666-416c-9399-5dac51c0efe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:13.443679 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 697/1390] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:24:13 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:24:13.691099 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47a892b-c651-4cb3-81aa-a0790306b0e4 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:13.691388 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47a892b-c651-4cb3-81aa-a0790306b0e4 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:13.691668 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47a892b-c651-4cb3-81aa-a0790306b0e4 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:13.693212 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 694/1391] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:24:13 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:24:13.698262 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85e5981-f615-440c-bb5c-8b63e41ffe21 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:13.698557 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85e5981-f615-440c-bb5c-8b63e41ffe21 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:13.698837 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85e5981-f615-440c-bb5c-8b63e41ffe21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:13.715555 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:13.730966 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47a892b-c651-4cb3-81aa-a0790306b0e4 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:13.732503 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-a47a892b-c651-4cb3-81aa-a0790306b0e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:13.764953 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:13.769943 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-a85e5981-f615-440c-bb5c-8b63e41ffe21 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:24:13.771171 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-a85e5981-f615-440c-bb5c-8b63e41ffe21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:13.892729 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 698/1392] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:24:13 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:24:13.894034 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47a892b-c651-4cb3-81aa-a0790306b0e4 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:13.995055 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:14.007457 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9714a00c-21a4-4764-abf2-c622541875ba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f8293316baf4774b62fc11d55d50865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:14.008061 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9714a00c-21a4-4764-abf2-c622541875ba service nova] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:14.008339 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9714a00c-21a4-4764-abf2-c622541875ba service nova] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:14.008612 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9714a00c-21a4-4764-abf2-c622541875ba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:14.010327 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-9714a00c-21a4-4764-abf2-c622541875ba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:14.101190 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9714a00c-21a4-4764-abf2-c622541875ba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:14.102838 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9714a00c-21a4-4764-abf2-c622541875ba service nova] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:14.130649 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 695/1393] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:24:13 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 17:24:14.198730 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dda5595-133d-4af4-989a-3ee80ec73661 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:14.199205 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dda5595-133d-4af4-989a-3ee80ec73661 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:14.199616 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dda5595-133d-4af4-989a-3ee80ec73661 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:14.201175 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 699/1394] 10.222.0.102 () {54 vars in 972 bytes} [Thu Sep 1 17:24:14 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:24:14.207433 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3558db4-908c-4d1d-815f-c7541c5a68a2 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:14.207819 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3558db4-908c-4d1d-815f-c7541c5a68a2 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:14.208157 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3558db4-908c-4d1d-815f-c7541c5a68a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:14.298347 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-e3558db4-908c-4d1d-815f-c7541c5a68a2 None None] MFA Rules not processed for user `039683b8d1d146c6add5ffa235d06f91`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:24:14.299592 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-e3558db4-908c-4d1d-815f-c7541c5a68a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:14.362157 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:14.388034 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 696/1395] 10.222.0.102 () {58 vars in 1069 bytes} [Thu Sep 1 17:24:14 2022] POST /identity/v3/auth/tokens => generated 3130 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:24:14.396150 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dda5595-133d-4af4-989a-3ee80ec73661 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:14.397889 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-3dda5595-133d-4af4-989a-3ee80ec73661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:14.500818 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:14.509539 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dda5595-133d-4af4-989a-3ee80ec73661 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:14.589837 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ed6e97-06d2-435e-aa4f-0e0eaadb0eb4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '039683b8d1d146c6add5ffa235d06f91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:14.590364 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ed6e97-06d2-435e-aa4f-0e0eaadb0eb4 service glance] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:14.590636 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ed6e97-06d2-435e-aa4f-0e0eaadb0eb4 service glance] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:14.590907 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ed6e97-06d2-435e-aa4f-0e0eaadb0eb4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:14.592530 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-d9ed6e97-06d2-435e-aa4f-0e0eaadb0eb4 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:14.702795 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ed6e97-06d2-435e-aa4f-0e0eaadb0eb4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:14.704204 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ed6e97-06d2-435e-aa4f-0e0eaadb0eb4 service glance] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:14.722670 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 700/1396] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:24:14 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 17:24:15.469592 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b39e97-a9c8-493b-81cb-ae9474bd3d80 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:15.470127 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b39e97-a9c8-493b-81cb-ae9474bd3d80 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:15.470127 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b39e97-a9c8-493b-81cb-ae9474bd3d80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:15.538915 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-f8b39e97-a9c8-493b-81cb-ae9474bd3d80 None None] MFA Rules not processed for user `575a76b99ea945efb1024fbb3dc81aaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:24:15.540130 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-f8b39e97-a9c8-493b-81cb-ae9474bd3d80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:15.628874 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 697/1397] 10.222.0.102 () {58 vars in 1014 bytes} [Thu Sep 1 17:24:15 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:24:15.680765 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e93893e-eeca-4456-b3c7-5d47fc377b8d None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:15.681281 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e93893e-eeca-4456-b3c7-5d47fc377b8d None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:15.681577 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e93893e-eeca-4456-b3c7-5d47fc377b8d None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:15.683009 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 701/1398] 10.222.0.102 () {54 vars in 948 bytes} [Thu Sep 1 17:24:15 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:24:15.689313 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99014106-c15d-4181-98a0-c81b5baf01c2 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:15.689619 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99014106-c15d-4181-98a0-c81b5baf01c2 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:15.689893 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99014106-c15d-4181-98a0-c81b5baf01c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:15.778780 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-99014106-c15d-4181-98a0-c81b5baf01c2 None None] MFA Rules not processed for user `d55edf6cd1804006aa8269b4c6f80aec`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:24:15.779913 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-99014106-c15d-4181-98a0-c81b5baf01c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:15.795248 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:15.919918 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 698/1399] 10.222.0.102 () {58 vars in 1045 bytes} [Thu Sep 1 17:24:15 2022] POST /identity/v3/auth/tokens => generated 3314 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:24:15.930032 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e93893e-eeca-4456-b3c7-5d47fc377b8d None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:15.931688 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-6e93893e-eeca-4456-b3c7-5d47fc377b8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:15.960462 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:16.109981 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e93893e-eeca-4456-b3c7-5d47fc377b8d None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:16.150379 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77fcf2c-ee70-4ddc-9b7a-a489913a3453 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:16.150698 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77fcf2c-ee70-4ddc-9b7a-a489913a3453 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:16.150982 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77fcf2c-ee70-4ddc-9b7a-a489913a3453 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:16.152371 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 699/1400] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:24:16 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Sep 01 17:24:16.157608 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ad2149-ef5f-48b5-96ae-99fd310f42bb None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:16.157893 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ad2149-ef5f-48b5-96ae-99fd310f42bb None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:16.158168 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ad2149-ef5f-48b5-96ae-99fd310f42bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:16.215063 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23db4d07-211a-4528-a80c-3c91d12ce940 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd55edf6cd1804006aa8269b4c6f80aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:16.215668 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23db4d07-211a-4528-a80c-3c91d12ce940 service swift] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:16.215945 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23db4d07-211a-4528-a80c-3c91d12ce940 service swift] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:16.216227 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23db4d07-211a-4528-a80c-3c91d12ce940 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:16.217915 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-23db4d07-211a-4528-a80c-3c91d12ce940 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:16.226309 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:16.228360 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-f7ad2149-ef5f-48b5-96ae-99fd310f42bb None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:24:16.229605 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-f7ad2149-ef5f-48b5-96ae-99fd310f42bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:16.308280 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:16.310147 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23db4d07-211a-4528-a80c-3c91d12ce940 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:16.311788 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23db4d07-211a-4528-a80c-3c91d12ce940 service swift] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:16.320174 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 702/1401] 10.222.0.102 () {60 vars in 1389 bytes} [Thu Sep 1 17:24:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 17:24:16.340553 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 700/1402] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:24:16 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:24:16.635472 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23db4d07-211a-4528-a80c-3c91d12ce940 service swift] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:16.636698 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-23db4d07-211a-4528-a80c-3c91d12ce940 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:16.741507 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23db4d07-211a-4528-a80c-3c91d12ce940 service swift] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:16.815923 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:16.885248 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdc5c05-c8a9-4ecb-b3bc-210fa437cd3d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f8293316baf4774b62fc11d55d50865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:16.885813 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdc5c05-c8a9-4ecb-b3bc-210fa437cd3d service nova] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:16.886089 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdc5c05-c8a9-4ecb-b3bc-210fa437cd3d service nova] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:16.886359 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdc5c05-c8a9-4ecb-b3bc-210fa437cd3d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:16.887969 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-abdc5c05-c8a9-4ecb-b3bc-210fa437cd3d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:17.003202 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdc5c05-c8a9-4ecb-b3bc-210fa437cd3d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:17.004792 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdc5c05-c8a9-4ecb-b3bc-210fa437cd3d service nova] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:17.028148 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 703/1403] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:24:16 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:24:19.030149 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1691e3a-88a7-4e87-bfc7-209fe54f8c1e None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:19.031168 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1691e3a-88a7-4e87-bfc7-209fe54f8c1e None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:19.031447 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1691e3a-88a7-4e87-bfc7-209fe54f8c1e None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:19.032849 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 701/1404] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:24:19 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:24:19.038598 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90888054-804e-423e-9f0e-edf676cee8cb None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:19.039091 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90888054-804e-423e-9f0e-edf676cee8cb None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:19.039183 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90888054-804e-423e-9f0e-edf676cee8cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:19.119462 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-90888054-804e-423e-9f0e-edf676cee8cb None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:24:19.120641 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-90888054-804e-423e-9f0e-edf676cee8cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:19.136071 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:19.234769 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 704/1405] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:24:19 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:24:19.527407 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1691e3a-88a7-4e87-bfc7-209fe54f8c1e None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:19.529780 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-e1691e3a-88a7-4e87-bfc7-209fe54f8c1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:19.709727 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1691e3a-88a7-4e87-bfc7-209fe54f8c1e None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:19.778006 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:19.868790 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9e44bf-58b6-4f68-9179-601a007fd19f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f8293316baf4774b62fc11d55d50865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:19.869490 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9e44bf-58b6-4f68-9179-601a007fd19f service nova] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:19.869923 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9e44bf-58b6-4f68-9179-601a007fd19f service nova] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:19.870203 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9e44bf-58b6-4f68-9179-601a007fd19f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:19.871885 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-6f9e44bf-58b6-4f68-9179-601a007fd19f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:19.978292 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9e44bf-58b6-4f68-9179-601a007fd19f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:19.980000 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9e44bf-58b6-4f68-9179-601a007fd19f service nova] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:19.998884 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 702/1406] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:24:19 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:24:22.033663 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0ab759-ecd3-4d85-9778-02203d1b1de5 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:22.034243 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0ab759-ecd3-4d85-9778-02203d1b1de5 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:22.034243 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0ab759-ecd3-4d85-9778-02203d1b1de5 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:22.035548 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 705/1407] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:24:22 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:24:22.042367 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960e4b17-ef43-4b7e-9dbf-81aedffea9ba None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:22.042867 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960e4b17-ef43-4b7e-9dbf-81aedffea9ba None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:22.042867 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960e4b17-ef43-4b7e-9dbf-81aedffea9ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:22.117363 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-960e4b17-ef43-4b7e-9dbf-81aedffea9ba None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:24:22.118816 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-960e4b17-ef43-4b7e-9dbf-81aedffea9ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:22.141418 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:22.235576 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 703/1408] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:24:22 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:24:22.542534 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0ab759-ecd3-4d85-9778-02203d1b1de5 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:22.544115 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-bf0ab759-ecd3-4d85-9778-02203d1b1de5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:22.771762 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:22.774799 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0ab759-ecd3-4d85-9778-02203d1b1de5 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:22.908399 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6080faa2-3946-41d7-9dcb-01043972d709 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f8293316baf4774b62fc11d55d50865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:22.909723 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6080faa2-3946-41d7-9dcb-01043972d709 service nova] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:22.910101 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6080faa2-3946-41d7-9dcb-01043972d709 service nova] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:22.910309 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6080faa2-3946-41d7-9dcb-01043972d709 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:22.912311 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-6080faa2-3946-41d7-9dcb-01043972d709 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:23.020143 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6080faa2-3946-41d7-9dcb-01043972d709 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:23.022577 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6080faa2-3946-41d7-9dcb-01043972d709 service nova] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:23.043112 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 706/1409] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:24:22 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 501 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:24:32.233600 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=35 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:33.040174 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=99 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:38.684886 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c18562-e693-4917-8d6b-11492a787841 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:38.685480 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c18562-e693-4917-8d6b-11492a787841 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:38.685581 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c18562-e693-4917-8d6b-11492a787841 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:38.686877 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 704/1410] 10.222.0.102 () {54 vars in 923 bytes} [Thu Sep 1 17:24:38 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:24:38.692592 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f89af51-c141-4607-a603-2a328528f607 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:38.693049 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f89af51-c141-4607-a603-2a328528f607 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:38.693154 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f89af51-c141-4607-a603-2a328528f607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:38.771794 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-5f89af51-c141-4607-a603-2a328528f607 None None] MFA Rules not processed for user `134ef6039cef468bae0e3f3be79b8073`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:24:38.773029 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-5f89af51-c141-4607-a603-2a328528f607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:38.881202 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 707/1411] 10.222.0.102 () {58 vars in 1020 bytes} [Thu Sep 1 17:24:38 2022] POST /identity/v3/auth/tokens => generated 3318 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:24:38.895385 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f250f96d-7a96-42a3-b6ad-6f87703f9581 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:38.895671 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f250f96d-7a96-42a3-b6ad-6f87703f9581 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:38.895945 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f250f96d-7a96-42a3-b6ad-6f87703f9581 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:38.897369 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 705/1412] 10.222.0.102 () {54 vars in 923 bytes} [Thu Sep 1 17:24:38 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:24:38.902043 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c114a65-50bf-4c43-9251-bcbc98634ff2 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:38.902333 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c114a65-50bf-4c43-9251-bcbc98634ff2 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:38.902609 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c114a65-50bf-4c43-9251-bcbc98634ff2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:38.977900 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:38.980521 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-7c114a65-50bf-4c43-9251-bcbc98634ff2 None None] MFA Rules not processed for user `5f8293316baf4774b62fc11d55d50865`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:24:38.982315 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-7c114a65-50bf-4c43-9251-bcbc98634ff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:39.111252 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 708/1413] 10.222.0.102 () {58 vars in 1020 bytes} [Thu Sep 1 17:24:38 2022] POST /identity/v3/auth/tokens => generated 3382 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:24:41.481673 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c4f17e-88c1-45b6-9dc2-ba024e00ba5b None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:41.482191 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c4f17e-88c1-45b6-9dc2-ba024e00ba5b None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:41.482191 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c4f17e-88c1-45b6-9dc2-ba024e00ba5b None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:41.483503 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 706/1414] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 17:24:41 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:24:41.489348 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a0299c-bfa2-4701-a69a-8f1ae9b24915 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:41.489768 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a0299c-bfa2-4701-a69a-8f1ae9b24915 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:41.489895 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a0299c-bfa2-4701-a69a-8f1ae9b24915 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:41.554590 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-c1a0299c-bfa2-4701-a69a-8f1ae9b24915 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:24:41.555807 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-c1a0299c-bfa2-4701-a69a-8f1ae9b24915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:41.611693 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:41.653748 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 709/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 17:24:41 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:24:41.945230 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c4f17e-88c1-45b6-9dc2-ba024e00ba5b None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:41.947243 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-a8c4f17e-88c1-45b6-9dc2-ba024e00ba5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:42.109905 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c4f17e-88c1-45b6-9dc2-ba024e00ba5b None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:42.182544 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:42.228301 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5f6f57-d83d-4aac-ac6f-f332366c018e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f8293316baf4774b62fc11d55d50865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:42.228913 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5f6f57-d83d-4aac-ac6f-f332366c018e service nova] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:42.229195 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5f6f57-d83d-4aac-ac6f-f332366c018e service nova] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:42.229501 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5f6f57-d83d-4aac-ac6f-f332366c018e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:42.231126 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-6e5f6f57-d83d-4aac-ac6f-f332366c018e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:42.328399 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5f6f57-d83d-4aac-ac6f-f332366c018e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:42.330060 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5f6f57-d83d-4aac-ac6f-f332366c018e service nova] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:42.351220 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 707/1416] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:24:41 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:24:43.245116 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907629aa-ef8c-495b-9e84-98a4c69b8ace None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:43.245669 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907629aa-ef8c-495b-9e84-98a4c69b8ace None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:43.245669 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907629aa-ef8c-495b-9e84-98a4c69b8ace None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:43.246960 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 710/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 17:24:43 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:24:43.251825 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ffb220-2923-4aa4-8941-7e639b9efa95 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:43.252230 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ffb220-2923-4aa4-8941-7e639b9efa95 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:43.252377 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ffb220-2923-4aa4-8941-7e639b9efa95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:43.326953 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-78ffb220-2923-4aa4-8941-7e639b9efa95 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:24:43.328178 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-78ffb220-2923-4aa4-8941-7e639b9efa95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:43.357000 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:43.425681 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 708/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 17:24:43 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:24:43.576416 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907629aa-ef8c-495b-9e84-98a4c69b8ace None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:43.577988 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-907629aa-ef8c-495b-9e84-98a4c69b8ace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:43.724621 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907629aa-ef8c-495b-9e84-98a4c69b8ace None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:43.774135 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:43.847542 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03f7fb6-38aa-46f9-a424-90addfd0852f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f8293316baf4774b62fc11d55d50865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:43.848728 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03f7fb6-38aa-46f9-a424-90addfd0852f service nova] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:43.849000 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03f7fb6-38aa-46f9-a424-90addfd0852f service nova] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:43.849264 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03f7fb6-38aa-46f9-a424-90addfd0852f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:43.850980 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-b03f7fb6-38aa-46f9-a424-90addfd0852f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:43.938760 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03f7fb6-38aa-46f9-a424-90addfd0852f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:43.940333 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03f7fb6-38aa-46f9-a424-90addfd0852f service nova] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:43.959496 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 711/1419] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:24:43 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:24:44.841881 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1e4c1f-db47-4f52-8743-cb3cdbf06164 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:44.842390 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1e4c1f-db47-4f52-8743-cb3cdbf06164 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:44.842493 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1e4c1f-db47-4f52-8743-cb3cdbf06164 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:44.843830 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 709/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 17:24:44 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:24:44.848750 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6113a65c-96aa-403c-a660-4bfee34a5985 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:44.849172 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6113a65c-96aa-403c-a660-4bfee34a5985 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:44.849319 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6113a65c-96aa-403c-a660-4bfee34a5985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:44.904134 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:44.925560 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-6113a65c-96aa-403c-a660-4bfee34a5985 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:24:44.926778 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-6113a65c-96aa-403c-a660-4bfee34a5985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:45.023880 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 712/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 17:24:44 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:24:45.174293 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1e4c1f-db47-4f52-8743-cb3cdbf06164 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:45.175820 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-0f1e4c1f-db47-4f52-8743-cb3cdbf06164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:45.322366 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1e4c1f-db47-4f52-8743-cb3cdbf06164 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:45.334793 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:45.438152 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a212e2ec-ab35-4b18-831d-686603239869 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f8293316baf4774b62fc11d55d50865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:45.438774 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a212e2ec-ab35-4b18-831d-686603239869 service nova] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:45.439058 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a212e2ec-ab35-4b18-831d-686603239869 service nova] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:45.439335 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a212e2ec-ab35-4b18-831d-686603239869 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:45.440956 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-a212e2ec-ab35-4b18-831d-686603239869 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:45.530319 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a212e2ec-ab35-4b18-831d-686603239869 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:45.531921 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a212e2ec-ab35-4b18-831d-686603239869 service nova] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:45.550171 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 710/1422] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:24:45 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:24:46.407820 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66361641-3ab1-4451-8bfb-de52bf5bf9b8 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:46.408396 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66361641-3ab1-4451-8bfb-de52bf5bf9b8 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:46.408396 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66361641-3ab1-4451-8bfb-de52bf5bf9b8 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:46.409746 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 713/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 17:24:46 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:24:46.414778 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d33678-f69b-409c-9a99-fe00b6411021 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:46.415215 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d33678-f69b-409c-9a99-fe00b6411021 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:46.415325 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d33678-f69b-409c-9a99-fe00b6411021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:46.434833 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:46.493087 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-42d33678-f69b-409c-9a99-fe00b6411021 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:24:46.494344 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-42d33678-f69b-409c-9a99-fe00b6411021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:46.590909 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 711/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 17:24:46 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:24:46.740281 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66361641-3ab1-4451-8bfb-de52bf5bf9b8 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:46.741785 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-66361641-3ab1-4451-8bfb-de52bf5bf9b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:46.853697 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:46.890113 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66361641-3ab1-4451-8bfb-de52bf5bf9b8 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:47.001761 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6eaff1f-32a6-4c25-96ee-8b0c7340e097 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f8293316baf4774b62fc11d55d50865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:47.002285 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eaff1f-32a6-4c25-96ee-8b0c7340e097 service nova] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:47.002563 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eaff1f-32a6-4c25-96ee-8b0c7340e097 service nova] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:47.002831 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eaff1f-32a6-4c25-96ee-8b0c7340e097 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:47.004446 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-f6eaff1f-32a6-4c25-96ee-8b0c7340e097 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:47.080884 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:47.095359 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6eaff1f-32a6-4c25-96ee-8b0c7340e097 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:47.096959 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6eaff1f-32a6-4c25-96ee-8b0c7340e097 service nova] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:47.115790 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 714/1425] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:24:46 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:24:47.984908 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66445d3-236a-430e-b369-65692d15658c None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:47.985499 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66445d3-236a-430e-b369-65692d15658c None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:47.985619 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66445d3-236a-430e-b369-65692d15658c None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:47.986940 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 712/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 17:24:47 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:24:47.992697 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037be8b9-042a-4a0d-97de-83b96b20776b None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:47.993124 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037be8b9-042a-4a0d-97de-83b96b20776b None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:47.993243 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037be8b9-042a-4a0d-97de-83b96b20776b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:48.068661 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-037be8b9-042a-4a0d-97de-83b96b20776b None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:24:48.069842 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-037be8b9-042a-4a0d-97de-83b96b20776b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:48.166680 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 715/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 17:24:47 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:24:48.316152 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66445d3-236a-430e-b369-65692d15658c None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:48.317751 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-a66445d3-236a-430e-b369-65692d15658c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:48.402293 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:48.471509 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66445d3-236a-430e-b369-65692d15658c None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:48.581909 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1fede1-2b53-4beb-867e-582d250b971a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f8293316baf4774b62fc11d55d50865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:48.582511 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1fede1-2b53-4beb-867e-582d250b971a service nova] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:48.583314 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1fede1-2b53-4beb-867e-582d250b971a service nova] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:48.583583 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1fede1-2b53-4beb-867e-582d250b971a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:48.585461 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-fb1fede1-2b53-4beb-867e-582d250b971a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:48.632729 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:48.675428 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1fede1-2b53-4beb-867e-582d250b971a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:48.676973 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1fede1-2b53-4beb-867e-582d250b971a service nova] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:48.695276 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 713/1428] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:24:48 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:24:49.508365 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ee9d2e-d653-405a-a8d7-65f6ffaf77ff None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:49.508947 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ee9d2e-d653-405a-a8d7-65f6ffaf77ff None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:49.508947 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ee9d2e-d653-405a-a8d7-65f6ffaf77ff None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:49.510287 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 716/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 17:24:49 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:24:49.515248 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e214e58c-0acb-4c07-a5b5-e8087bb19b60 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:49.515689 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e214e58c-0acb-4c07-a5b5-e8087bb19b60 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:49.515785 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e214e58c-0acb-4c07-a5b5-e8087bb19b60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:49.591014 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-e214e58c-0acb-4c07-a5b5-e8087bb19b60 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:24:49.592236 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-e214e58c-0acb-4c07-a5b5-e8087bb19b60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:49.690477 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 714/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 17:24:49 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:24:49.845281 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ee9d2e-d653-405a-a8d7-65f6ffaf77ff None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:49.846838 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-83ee9d2e-d653-405a-a8d7-65f6ffaf77ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:49.896632 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:49.999428 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ee9d2e-d653-405a-a8d7-65f6ffaf77ff None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:50.125583 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c039b8-76b8-4d82-8643-eadc4d949992 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f8293316baf4774b62fc11d55d50865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:50.126104 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c039b8-76b8-4d82-8643-eadc4d949992 service nova] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:50.126383 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c039b8-76b8-4d82-8643-eadc4d949992 service nova] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:50.126656 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c039b8-76b8-4d82-8643-eadc4d949992 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:50.128271 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-68c039b8-76b8-4d82-8643-eadc4d949992 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:50.135852 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:50.218856 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c039b8-76b8-4d82-8643-eadc4d949992 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:50.220453 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c039b8-76b8-4d82-8643-eadc4d949992 service nova] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:50.240113 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 717/1431] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:24:49 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:24:51.049471 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac8a6fd-ef0f-4dd3-a6c2-8e65d995ab69 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:51.050013 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac8a6fd-ef0f-4dd3-a6c2-8e65d995ab69 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:51.050013 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac8a6fd-ef0f-4dd3-a6c2-8e65d995ab69 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:51.051388 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 715/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 17:24:51 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:24:51.056362 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59ed2c2-cc62-45ad-8aaa-cb43eb535f08 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:51.056813 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59ed2c2-cc62-45ad-8aaa-cb43eb535f08 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:51.056919 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59ed2c2-cc62-45ad-8aaa-cb43eb535f08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:51.133511 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-c59ed2c2-cc62-45ad-8aaa-cb43eb535f08 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:24:51.134741 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-c59ed2c2-cc62-45ad-8aaa-cb43eb535f08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:51.203338 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:51.233455 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 718/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 17:24:51 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:24:51.393244 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac8a6fd-ef0f-4dd3-a6c2-8e65d995ab69 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:51.394717 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-cac8a6fd-ef0f-4dd3-a6c2-8e65d995ab69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:51.412808 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:51.541290 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac8a6fd-ef0f-4dd3-a6c2-8e65d995ab69 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:51.630973 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:51.657890 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e68334-a70d-4bd1-baf2-246af8ade3a3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f8293316baf4774b62fc11d55d50865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:51.658422 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e68334-a70d-4bd1-baf2-246af8ade3a3 service nova] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:51.658712 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e68334-a70d-4bd1-baf2-246af8ade3a3 service nova] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:51.658977 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e68334-a70d-4bd1-baf2-246af8ade3a3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:51.660585 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-d2e68334-a70d-4bd1-baf2-246af8ade3a3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:51.750477 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e68334-a70d-4bd1-baf2-246af8ade3a3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:51.752105 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e68334-a70d-4bd1-baf2-246af8ade3a3 service nova] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:51.770538 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 716/1434] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:24:51 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:24:52.579457 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd152e38-b06e-4241-9f54-9db1fe78feee None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:52.580017 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd152e38-b06e-4241-9f54-9db1fe78feee None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:52.580017 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd152e38-b06e-4241-9f54-9db1fe78feee None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:52.581537 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 719/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 17:24:52 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:24:52.587416 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3413c63b-de3c-4a65-89c8-fa69e75315b6 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:52.587852 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3413c63b-de3c-4a65-89c8-fa69e75315b6 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:52.587973 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3413c63b-de3c-4a65-89c8-fa69e75315b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:52.663904 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-3413c63b-de3c-4a65-89c8-fa69e75315b6 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:24:52.665166 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-3413c63b-de3c-4a65-89c8-fa69e75315b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:52.707450 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:52.764482 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 717/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 17:24:52 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:24:52.915919 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd152e38-b06e-4241-9f54-9db1fe78feee None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:52.917522 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-fd152e38-b06e-4241-9f54-9db1fe78feee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:53.062394 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd152e38-b06e-4241-9f54-9db1fe78feee None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:53.117379 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:53.175684 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6376798-10db-4090-aa7a-ce6c41cb4be8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f8293316baf4774b62fc11d55d50865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:53.176287 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6376798-10db-4090-aa7a-ce6c41cb4be8 service nova] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:53.176562 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6376798-10db-4090-aa7a-ce6c41cb4be8 service nova] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:53.176856 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6376798-10db-4090-aa7a-ce6c41cb4be8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:53.178493 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-f6376798-10db-4090-aa7a-ce6c41cb4be8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:53.266805 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6376798-10db-4090-aa7a-ce6c41cb4be8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:53.268393 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6376798-10db-4090-aa7a-ce6c41cb4be8 service nova] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:53.286295 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 720/1437] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:24:52 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:24:54.094825 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d85037d-3b84-4936-81b5-606a44489157 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:54.095387 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d85037d-3b84-4936-81b5-606a44489157 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:54.095387 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d85037d-3b84-4936-81b5-606a44489157 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:54.096769 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 718/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 17:24:54 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:24:54.102319 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dd5d8a-7d2a-46a8-99b3-d07d0255c6e3 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:54.102769 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dd5d8a-7d2a-46a8-99b3-d07d0255c6e3 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:54.102882 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dd5d8a-7d2a-46a8-99b3-d07d0255c6e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:54.179683 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-08dd5d8a-7d2a-46a8-99b3-d07d0255c6e3 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:24:54.180900 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-08dd5d8a-7d2a-46a8-99b3-d07d0255c6e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:54.182976 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:54.277614 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 721/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 17:24:54 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:24:54.426309 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d85037d-3b84-4936-81b5-606a44489157 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:54.427839 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-0d85037d-3b84-4936-81b5-606a44489157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:54.573428 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d85037d-3b84-4936-81b5-606a44489157 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:54.600613 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:54.686248 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49f0c37-cc09-4a5d-89df-bb1d018b7f15 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f8293316baf4774b62fc11d55d50865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:54.686776 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49f0c37-cc09-4a5d-89df-bb1d018b7f15 service nova] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:54.687056 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49f0c37-cc09-4a5d-89df-bb1d018b7f15 service nova] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:54.687326 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49f0c37-cc09-4a5d-89df-bb1d018b7f15 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:54.688942 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-a49f0c37-cc09-4a5d-89df-bb1d018b7f15 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:54.786424 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49f0c37-cc09-4a5d-89df-bb1d018b7f15 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:54.788038 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49f0c37-cc09-4a5d-89df-bb1d018b7f15 service nova] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:54.806518 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 719/1440] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:24:54 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:24:56.624209 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f7d003-052b-404b-9b49-721afe9d85e5 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:56.624777 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f7d003-052b-404b-9b49-721afe9d85e5 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:56.624777 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f7d003-052b-404b-9b49-721afe9d85e5 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:56.626154 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 722/1441] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 17:24:56 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:24:56.631265 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c21cda-c470-486b-8196-d937703531ac None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:56.631655 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c21cda-c470-486b-8196-d937703531ac None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:56.631814 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c21cda-c470-486b-8196-d937703531ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:56.672935 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:56.699145 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-27c21cda-c470-486b-8196-d937703531ac None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:24:56.700391 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-27c21cda-c470-486b-8196-d937703531ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:56.796840 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 720/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 17:24:56 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:24:57.082081 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f7d003-052b-404b-9b49-721afe9d85e5 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:57.084212 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-93f7d003-052b-404b-9b49-721afe9d85e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:57.187440 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f7d003-052b-404b-9b49-721afe9d85e5 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:57.220352 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:57.262720 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637d5686-4adc-4cf2-82b7-81c882854c8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:57.263335 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637d5686-4adc-4cf2-82b7-81c882854c8c service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:57.263616 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637d5686-4adc-4cf2-82b7-81c882854c8c service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:57.263888 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637d5686-4adc-4cf2-82b7-81c882854c8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:57.265535 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-637d5686-4adc-4cf2-82b7-81c882854c8c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:57.373944 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637d5686-4adc-4cf2-82b7-81c882854c8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:57.375291 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637d5686-4adc-4cf2-82b7-81c882854c8c service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:57.394013 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 723/1443] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:24:57 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:24:59.464443 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7444efed-d040-4a2a-bc91-ea3d739b72e4 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:59.465018 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7444efed-d040-4a2a-bc91-ea3d739b72e4 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:59.465018 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7444efed-d040-4a2a-bc91-ea3d739b72e4 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:59.466410 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 721/1444] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 17:24:59 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:24:59.471693 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbe91fb-a774-4f1f-a66f-51bf3db9093b None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:59.472121 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbe91fb-a774-4f1f-a66f-51bf3db9093b None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:59.472235 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbe91fb-a774-4f1f-a66f-51bf3db9093b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:59.537459 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-bdbe91fb-a774-4f1f-a66f-51bf3db9093b None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:24:59.538665 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-bdbe91fb-a774-4f1f-a66f-51bf3db9093b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:59.551268 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:59.636791 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 724/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 17:24:59 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:24:59.919373 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7444efed-d040-4a2a-bc91-ea3d739b72e4 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:59.920916 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-7444efed-d040-4a2a-bc91-ea3d739b72e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:25:00.043776 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7444efed-d040-4a2a-bc91-ea3d739b72e4 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:25:00.055986 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:25:00.119813 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc72b01-8bbf-4af2-90c0-6d39af8e7f66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:25:00.120335 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc72b01-8bbf-4af2-90c0-6d39af8e7f66 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:00.120612 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc72b01-8bbf-4af2-90c0-6d39af8e7f66 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:00.120884 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc72b01-8bbf-4af2-90c0-6d39af8e7f66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:00.122557 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-0fc72b01-8bbf-4af2-90c0-6d39af8e7f66 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:25:00.240313 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc72b01-8bbf-4af2-90c0-6d39af8e7f66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:25:00.241821 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc72b01-8bbf-4af2-90c0-6d39af8e7f66 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:25:00.259724 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 722/1446] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:24:59 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:25:02.534057 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda99490-e23c-4f3e-a211-857ddeb8ac24 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:02.534625 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda99490-e23c-4f3e-a211-857ddeb8ac24 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:02.534625 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda99490-e23c-4f3e-a211-857ddeb8ac24 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:02.535957 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 725/1447] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 17:25:02 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:25:02.541216 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241bbccf-8e51-4389-a5d9-2dd3d6befba4 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:02.541658 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241bbccf-8e51-4389-a5d9-2dd3d6befba4 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:02.541804 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241bbccf-8e51-4389-a5d9-2dd3d6befba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:02.591912 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:25:02.610437 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-241bbccf-8e51-4389-a5d9-2dd3d6befba4 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:25:02.611686 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-241bbccf-8e51-4389-a5d9-2dd3d6befba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:25:02.712198 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 723/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 17:25:02 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:25:02.997555 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda99490-e23c-4f3e-a211-857ddeb8ac24 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:25:02.999151 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-eda99490-e23c-4f3e-a211-857ddeb8ac24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:25:03.101664 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda99490-e23c-4f3e-a211-857ddeb8ac24 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:25:03.140446 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:25:03.175237 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12635c30-acac-4d0f-b16c-431a267f387b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:25:03.175839 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12635c30-acac-4d0f-b16c-431a267f387b service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:03.176115 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12635c30-acac-4d0f-b16c-431a267f387b service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:03.176384 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12635c30-acac-4d0f-b16c-431a267f387b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:03.178031 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-12635c30-acac-4d0f-b16c-431a267f387b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:25:03.286933 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12635c30-acac-4d0f-b16c-431a267f387b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:25:03.288358 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12635c30-acac-4d0f-b16c-431a267f387b service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:25:03.308277 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 726/1449] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:25:02 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:25:05.338796 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b6a875-395f-47d1-9ca0-b62ae72a7077 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:05.339360 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b6a875-395f-47d1-9ca0-b62ae72a7077 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:05.339360 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b6a875-395f-47d1-9ca0-b62ae72a7077 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:05.340712 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 724/1450] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 17:25:05 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:25:05.345955 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c18d6be-e70d-4b52-9243-de427e99e8b9 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:05.346382 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c18d6be-e70d-4b52-9243-de427e99e8b9 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:05.346495 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c18d6be-e70d-4b52-9243-de427e99e8b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:05.412968 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-4c18d6be-e70d-4b52-9243-de427e99e8b9 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:25:05.414230 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-4c18d6be-e70d-4b52-9243-de427e99e8b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:25:05.434504 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:25:05.513442 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 727/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 17:25:05 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:25:05.792751 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b6a875-395f-47d1-9ca0-b62ae72a7077 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:25:05.794351 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-d0b6a875-395f-47d1-9ca0-b62ae72a7077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:25:05.911163 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b6a875-395f-47d1-9ca0-b62ae72a7077 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:25:05.931605 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:25:05.986935 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98347f7-073f-4650-87ea-6fdad8ebe6d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:25:05.987548 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98347f7-073f-4650-87ea-6fdad8ebe6d6 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:05.987830 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98347f7-073f-4650-87ea-6fdad8ebe6d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:05.988105 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98347f7-073f-4650-87ea-6fdad8ebe6d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:05.989821 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-e98347f7-073f-4650-87ea-6fdad8ebe6d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:25:06.109488 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98347f7-073f-4650-87ea-6fdad8ebe6d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:25:06.110927 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98347f7-073f-4650-87ea-6fdad8ebe6d6 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:25:06.130051 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 725/1452] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:25:05 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:25:08.198223 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d07e8d9-6230-4fcc-a84a-b90ed3a2f6ea None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:08.198762 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d07e8d9-6230-4fcc-a84a-b90ed3a2f6ea None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:08.198762 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d07e8d9-6230-4fcc-a84a-b90ed3a2f6ea None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:08.200054 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 728/1453] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 17:25:08 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:25:08.205428 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf45ca0-f05d-4fed-b1cc-c89285019f35 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:08.205830 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf45ca0-f05d-4fed-b1cc-c89285019f35 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:08.205992 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf45ca0-f05d-4fed-b1cc-c89285019f35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:08.270484 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:25:08.273543 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-bcf45ca0-f05d-4fed-b1cc-c89285019f35 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:25:08.275383 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-bcf45ca0-f05d-4fed-b1cc-c89285019f35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:25:08.388686 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 726/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 17:25:08 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:25:08.674861 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d07e8d9-6230-4fcc-a84a-b90ed3a2f6ea None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:25:08.676516 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-7d07e8d9-6230-4fcc-a84a-b90ed3a2f6ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:25:08.782856 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d07e8d9-6230-4fcc-a84a-b90ed3a2f6ea None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:25:08.835737 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:25:08.873689 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bae0aea-4372-416b-a6f0-68a599339ff8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:25:08.874680 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bae0aea-4372-416b-a6f0-68a599339ff8 service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:08.875148 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bae0aea-4372-416b-a6f0-68a599339ff8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:08.875608 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bae0aea-4372-416b-a6f0-68a599339ff8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:08.878532 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-6bae0aea-4372-416b-a6f0-68a599339ff8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:25:09.018577 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bae0aea-4372-416b-a6f0-68a599339ff8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:25:09.019955 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bae0aea-4372-416b-a6f0-68a599339ff8 service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:25:09.040332 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 729/1455] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:25:08 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:25:11.004035 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53139ff9-481f-45d5-8a24-3ef81fc924c5 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:11.004601 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53139ff9-481f-45d5-8a24-3ef81fc924c5 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:11.004601 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53139ff9-481f-45d5-8a24-3ef81fc924c5 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:11.005976 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 727/1456] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 17:25:11 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:25:11.011629 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d101909-ea1a-49a7-8165-0aacefb06983 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:11.012023 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d101909-ea1a-49a7-8165-0aacefb06983 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:11.012183 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d101909-ea1a-49a7-8165-0aacefb06983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:11.081083 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-1d101909-ea1a-49a7-8165-0aacefb06983 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:25:11.082492 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-1d101909-ea1a-49a7-8165-0aacefb06983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:25:11.113108 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:25:11.186200 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 730/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 17:25:11 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:25:11.470992 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53139ff9-481f-45d5-8a24-3ef81fc924c5 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:25:11.472633 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-53139ff9-481f-45d5-8a24-3ef81fc924c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:25:11.693585 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53139ff9-481f-45d5-8a24-3ef81fc924c5 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:25:11.721572 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:25:11.767793 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d98395c-de24-4221-a336-69aa8d73a928 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:25:11.768404 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d98395c-de24-4221-a336-69aa8d73a928 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:11.768681 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d98395c-de24-4221-a336-69aa8d73a928 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:11.768959 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d98395c-de24-4221-a336-69aa8d73a928 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:11.770594 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-0d98395c-de24-4221-a336-69aa8d73a928 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:25:11.881870 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d98395c-de24-4221-a336-69aa8d73a928 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:25:11.883309 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d98395c-de24-4221-a336-69aa8d73a928 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:25:11.901396 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 728/1458] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:25:11 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:25:13.963973 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58ae7a7-350f-458e-b697-06a8fc1d0231 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:13.964597 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58ae7a7-350f-458e-b697-06a8fc1d0231 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:13.964597 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58ae7a7-350f-458e-b697-06a8fc1d0231 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:13.965907 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 731/1459] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 17:25:13 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:25:13.973508 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da242255-4d60-4ed9-a7db-38c281c719d8 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:13.973925 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da242255-4d60-4ed9-a7db-38c281c719d8 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:13.974037 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da242255-4d60-4ed9-a7db-38c281c719d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:14.038584 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-da242255-4d60-4ed9-a7db-38c281c719d8 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:25:14.039781 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-da242255-4d60-4ed9-a7db-38c281c719d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:25:14.048209 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:25:14.137515 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 729/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 17:25:13 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:25:14.418691 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58ae7a7-350f-458e-b697-06a8fc1d0231 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:25:14.420238 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-c58ae7a7-350f-458e-b697-06a8fc1d0231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:25:14.523930 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58ae7a7-350f-458e-b697-06a8fc1d0231 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:25:14.578520 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:25:14.597554 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b914d451-938d-4329-ada0-c95bc1aa9462 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:25:14.598171 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b914d451-938d-4329-ada0-c95bc1aa9462 service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:14.598453 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b914d451-938d-4329-ada0-c95bc1aa9462 service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:14.598725 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b914d451-938d-4329-ada0-c95bc1aa9462 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:14.600934 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-b914d451-938d-4329-ada0-c95bc1aa9462 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:25:14.709398 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b914d451-938d-4329-ada0-c95bc1aa9462 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:25:14.711384 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b914d451-938d-4329-ada0-c95bc1aa9462 service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:25:14.730145 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 732/1461] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:25:14 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:25:16.699213 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d972b3-4ac4-4466-a78e-0ab761769be5 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:16.699785 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d972b3-4ac4-4466-a78e-0ab761769be5 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:16.699899 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d972b3-4ac4-4466-a78e-0ab761769be5 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:16.701216 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 730/1462] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 17:25:16 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:25:16.706615 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d0aaea-4340-4597-9778-e91ec6aa6909 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:16.707076 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d0aaea-4340-4597-9778-e91ec6aa6909 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:16.707294 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d0aaea-4340-4597-9778-e91ec6aa6909 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:16.773414 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-75d0aaea-4340-4597-9778-e91ec6aa6909 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:25:16.774625 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-75d0aaea-4340-4597-9778-e91ec6aa6909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:25:16.812515 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:25:16.872281 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 733/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 17:25:16 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:25:17.157080 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d972b3-4ac4-4466-a78e-0ab761769be5 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:25:17.158690 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-88d972b3-4ac4-4466-a78e-0ab761769be5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:25:17.264111 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d972b3-4ac4-4466-a78e-0ab761769be5 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:25:17.302267 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:25:17.337998 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d55b689-b02a-425f-8233-6482beee4522 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:25:17.339374 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d55b689-b02a-425f-8233-6482beee4522 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:17.339374 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d55b689-b02a-425f-8233-6482beee4522 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:17.339374 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d55b689-b02a-425f-8233-6482beee4522 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:17.340852 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-0d55b689-b02a-425f-8233-6482beee4522 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:25:17.449736 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d55b689-b02a-425f-8233-6482beee4522 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:25:17.451197 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d55b689-b02a-425f-8233-6482beee4522 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:25:17.469757 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 731/1464] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:25:17 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:25:19.415155 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b785830-e6e9-4bc6-9679-e8649a022a92 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:19.415817 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b785830-e6e9-4bc6-9679-e8649a022a92 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:19.415817 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b785830-e6e9-4bc6-9679-e8649a022a92 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:19.417079 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 734/1465] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 17:25:19 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:25:19.422511 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac132aee-a7ee-4217-b42a-a1b959f397cb None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:19.422969 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac132aee-a7ee-4217-b42a-a1b959f397cb None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:19.423085 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac132aee-a7ee-4217-b42a-a1b959f397cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:19.489205 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-ac132aee-a7ee-4217-b42a-a1b959f397cb None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:25:19.490450 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-ac132aee-a7ee-4217-b42a-a1b959f397cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:25:19.508905 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:25:19.587681 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 732/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 17:25:19 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:25:19.870123 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b785830-e6e9-4bc6-9679-e8649a022a92 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:25:19.871713 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-5b785830-e6e9-4bc6-9679-e8649a022a92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:25:19.976732 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b785830-e6e9-4bc6-9679-e8649a022a92 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:25:20.043194 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:25:20.051030 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43b3530-200c-4286-a790-0cc797d2e024 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:25:20.051638 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43b3530-200c-4286-a790-0cc797d2e024 service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:20.051910 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43b3530-200c-4286-a790-0cc797d2e024 service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:20.052173 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43b3530-200c-4286-a790-0cc797d2e024 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:20.053803 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-e43b3530-200c-4286-a790-0cc797d2e024 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:25:20.162417 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43b3530-200c-4286-a790-0cc797d2e024 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:25:20.163826 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43b3530-200c-4286-a790-0cc797d2e024 service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:25:20.182051 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 735/1467] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:25:19 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:25:22.125134 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bddb49-6e13-4ef2-9e34-1323c2c1f06f None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:22.125683 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bddb49-6e13-4ef2-9e34-1323c2c1f06f None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:22.125683 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bddb49-6e13-4ef2-9e34-1323c2c1f06f None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:22.127027 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 733/1468] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 17:25:22 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:25:22.132517 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba7f54e-1a4d-4ccd-af3b-4325a675495f None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:22.132926 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba7f54e-1a4d-4ccd-af3b-4325a675495f None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:22.133071 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba7f54e-1a4d-4ccd-af3b-4325a675495f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:22.198649 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-4ba7f54e-1a4d-4ccd-af3b-4325a675495f None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:25:22.199877 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-4ba7f54e-1a4d-4ccd-af3b-4325a675495f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:25:22.250134 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:25:22.299421 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 736/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 17:25:22 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:25:22.587576 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bddb49-6e13-4ef2-9e34-1323c2c1f06f None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:25:22.589431 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-33bddb49-6e13-4ef2-9e34-1323c2c1f06f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:25:22.690616 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bddb49-6e13-4ef2-9e34-1323c2c1f06f None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:25:22.739993 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:25:22.765684 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a389e53e-2a81-4680-bf80-41ab2e4ea926 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:25:22.766286 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a389e53e-2a81-4680-bf80-41ab2e4ea926 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:22.766563 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a389e53e-2a81-4680-bf80-41ab2e4ea926 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:22.766833 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a389e53e-2a81-4680-bf80-41ab2e4ea926 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:22.768431 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-a389e53e-2a81-4680-bf80-41ab2e4ea926 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:25:22.881749 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a389e53e-2a81-4680-bf80-41ab2e4ea926 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:25:22.884003 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a389e53e-2a81-4680-bf80-41ab2e4ea926 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:25:22.904343 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 734/1470] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:25:22 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:25:32.297632 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=22 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:25:32.902201 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=66 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:25:42.511329 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3227aba5-d821-473d-9649-dfa9499d89b7 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:42.511928 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3227aba5-d821-473d-9649-dfa9499d89b7 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:42.511928 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3227aba5-d821-473d-9649-dfa9499d89b7 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:42.513269 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 737/1471] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:25:42 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:25:42.519003 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740ae64c-2aa7-426f-b0c1-ec34595b1399 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:42.519495 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740ae64c-2aa7-426f-b0c1-ec34595b1399 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:42.519614 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740ae64c-2aa7-426f-b0c1-ec34595b1399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:42.585024 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-740ae64c-2aa7-426f-b0c1-ec34595b1399 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:25:42.586268 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-740ae64c-2aa7-426f-b0c1-ec34595b1399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:25:42.682245 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 735/1472] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:25:42 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:25:42.977847 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a1c474-e70b-4f20-aaa1-80d5737c1624 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:42.978137 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a1c474-e70b-4f20-aaa1-80d5737c1624 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:42.978418 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a1c474-e70b-4f20-aaa1-80d5737c1624 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:42.979764 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 738/1473] 10.222.0.102 () {54 vars in 972 bytes} [Thu Sep 1 17:25:42 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:25:42.984467 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94721ad8-3804-4a0a-886b-f35ca8b36908 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:42.984756 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94721ad8-3804-4a0a-886b-f35ca8b36908 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:42.985036 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94721ad8-3804-4a0a-886b-f35ca8b36908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:43.060843 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-94721ad8-3804-4a0a-886b-f35ca8b36908 None None] MFA Rules not processed for user `039683b8d1d146c6add5ffa235d06f91`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:25:43.062163 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-94721ad8-3804-4a0a-886b-f35ca8b36908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:25:43.083938 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:25:43.140963 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 736/1474] 10.222.0.102 () {58 vars in 1069 bytes} [Thu Sep 1 17:25:42 2022] POST /identity/v3/auth/tokens => generated 3130 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:25:43.148554 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a1c474-e70b-4f20-aaa1-80d5737c1624 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:25:43.150140 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-b1a1c474-e70b-4f20-aaa1-80d5737c1624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:25:43.256969 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a1c474-e70b-4f20-aaa1-80d5737c1624 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:25:43.330369 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb915f0-2b66-4048-ad81-e2328417d3c3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '039683b8d1d146c6add5ffa235d06f91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:25:43.330884 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb915f0-2b66-4048-ad81-e2328417d3c3 service glance] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:43.331160 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb915f0-2b66-4048-ad81-e2328417d3c3 service glance] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:43.331427 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb915f0-2b66-4048-ad81-e2328417d3c3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:43.333013 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-7fb915f0-2b66-4048-ad81-e2328417d3c3 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:25:43.387694 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:25:43.443372 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb915f0-2b66-4048-ad81-e2328417d3c3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:25:43.444813 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb915f0-2b66-4048-ad81-e2328417d3c3 service glance] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:25:43.463137 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 739/1475] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:25:43 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:25:45.481780 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2840eb1e-1e30-4367-ba51-8d51330e78b3 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:45.482326 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2840eb1e-1e30-4367-ba51-8d51330e78b3 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:45.482326 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2840eb1e-1e30-4367-ba51-8d51330e78b3 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:45.483734 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 737/1476] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:25:45 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:25:45.488938 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2971a3-a36a-4015-9fa9-a49960775bab None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:45.489383 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2971a3-a36a-4015-9fa9-a49960775bab None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:45.489522 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2971a3-a36a-4015-9fa9-a49960775bab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:45.558340 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-aa2971a3-a36a-4015-9fa9-a49960775bab None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:25:45.559560 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-aa2971a3-a36a-4015-9fa9-a49960775bab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:25:45.672015 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 740/1477] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:25:45 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:25:45.944069 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2840eb1e-1e30-4367-ba51-8d51330e78b3 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:25:45.945683 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-2840eb1e-1e30-4367-ba51-8d51330e78b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:25:46.073574 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2840eb1e-1e30-4367-ba51-8d51330e78b3 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:25:46.123499 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:25:46.169851 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2084a4-507b-4538-9604-e61877b44f16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:25:46.170467 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2084a4-507b-4538-9604-e61877b44f16 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:46.170750 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2084a4-507b-4538-9604-e61877b44f16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:46.171021 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2084a4-507b-4538-9604-e61877b44f16 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:46.171657 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2084a4-507b-4538-9604-e61877b44f16 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:25:46.173431 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: WARNING py.warnings [None req-da2084a4-507b-4538-9604-e61877b44f16 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 17:25:46.173431 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: warnings.warn(msg) Sep 01 17:25:46.173431 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: Sep 01 17:25:46.174581 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2084a4-507b-4538-9604-e61877b44f16 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:25:46.193102 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 738/1478] 10.222.0.102 () {56 vars in 1113 bytes} [Thu Sep 1 17:25:45 2022] GET /identity/v3/projects => generated 2562 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:25:48.133643 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a54c6a1-36b3-405e-8d57-34d0e0eb56f7 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:48.134213 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a54c6a1-36b3-405e-8d57-34d0e0eb56f7 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:48.134213 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a54c6a1-36b3-405e-8d57-34d0e0eb56f7 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:48.135654 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 741/1479] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:25:48 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:25:48.140974 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7586ce-81e2-42c3-8995-f925a978dfd2 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:48.141468 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7586ce-81e2-42c3-8995-f925a978dfd2 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:48.141573 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7586ce-81e2-42c3-8995-f925a978dfd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:48.207123 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-2c7586ce-81e2-42c3-8995-f925a978dfd2 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:25:48.208343 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-2c7586ce-81e2-42c3-8995-f925a978dfd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:25:48.307434 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 739/1480] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:25:48 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:25:48.618070 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a54c6a1-36b3-405e-8d57-34d0e0eb56f7 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:25:48.620202 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-8a54c6a1-36b3-405e-8d57-34d0e0eb56f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:25:48.630865 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:25:48.766284 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a54c6a1-36b3-405e-8d57-34d0e0eb56f7 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:25:48.847370 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:25:48.881756 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb967adb-d7ec-44bb-b04c-c8f38f38f9f6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f8293316baf4774b62fc11d55d50865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:25:48.882385 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb967adb-d7ec-44bb-b04c-c8f38f38f9f6 service nova] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:48.882669 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb967adb-d7ec-44bb-b04c-c8f38f38f9f6 service nova] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:48.882942 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb967adb-d7ec-44bb-b04c-c8f38f38f9f6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:48.884545 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-bb967adb-d7ec-44bb-b04c-c8f38f38f9f6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:25:48.980647 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb967adb-d7ec-44bb-b04c-c8f38f38f9f6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:25:48.982282 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb967adb-d7ec-44bb-b04c-c8f38f38f9f6 service nova] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:25:49.000099 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 742/1481] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:25:48 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:25:51.104562 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325662de-a749-485a-b5ce-df4e6abf79e4 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:51.105193 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325662de-a749-485a-b5ce-df4e6abf79e4 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:51.105193 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325662de-a749-485a-b5ce-df4e6abf79e4 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:51.106689 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 740/1482] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:25:51 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:25:51.111975 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d75353-3ad5-41cd-ab54-5afc80ba290e None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:51.112394 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d75353-3ad5-41cd-ab54-5afc80ba290e None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:51.112539 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d75353-3ad5-41cd-ab54-5afc80ba290e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:51.178425 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-51d75353-3ad5-41cd-ab54-5afc80ba290e None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:25:51.179653 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-51d75353-3ad5-41cd-ab54-5afc80ba290e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:25:51.226876 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:25:51.280261 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 743/1483] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:25:51 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:25:51.566238 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325662de-a749-485a-b5ce-df4e6abf79e4 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:25:51.567799 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-325662de-a749-485a-b5ce-df4e6abf79e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:25:51.591309 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:25:51.671727 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325662de-a749-485a-b5ce-df4e6abf79e4 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:25:51.747876 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1436266b-6f16-45c1-bf28-68e70d6b2515 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:25:51.748483 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1436266b-6f16-45c1-bf28-68e70d6b2515 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:51.748756 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1436266b-6f16-45c1-bf28-68e70d6b2515 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:51.749024 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1436266b-6f16-45c1-bf28-68e70d6b2515 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:51.750687 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-1436266b-6f16-45c1-bf28-68e70d6b2515 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:25:51.825689 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:25:51.861135 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1436266b-6f16-45c1-bf28-68e70d6b2515 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:25:51.862868 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1436266b-6f16-45c1-bf28-68e70d6b2515 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:25:51.881720 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 741/1484] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:25:51 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:25:53.851734 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03432aa-1d03-491c-8050-8050ee3bacbf None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:53.852287 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03432aa-1d03-491c-8050-8050ee3bacbf None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:53.852287 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03432aa-1d03-491c-8050-8050ee3bacbf None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:53.854463 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 744/1485] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:25:53 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:25:53.859753 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d828eb-09f5-43b3-b21f-54f279c43720 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:53.860165 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d828eb-09f5-43b3-b21f-54f279c43720 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:53.860319 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d828eb-09f5-43b3-b21f-54f279c43720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:53.927336 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-48d828eb-09f5-43b3-b21f-54f279c43720 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:25:53.928557 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-48d828eb-09f5-43b3-b21f-54f279c43720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:25:54.026151 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 742/1486] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:25:53 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:25:54.315841 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03432aa-1d03-491c-8050-8050ee3bacbf None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:25:54.317482 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-e03432aa-1d03-491c-8050-8050ee3bacbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:25:54.426585 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03432aa-1d03-491c-8050-8050ee3bacbf None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:25:54.499086 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:25:54.500952 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78503550-91a8-4c9c-99f9-2ed706b24ea9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:25:54.501604 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78503550-91a8-4c9c-99f9-2ed706b24ea9 service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:54.502064 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78503550-91a8-4c9c-99f9-2ed706b24ea9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:54.502382 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78503550-91a8-4c9c-99f9-2ed706b24ea9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:54.504316 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-78503550-91a8-4c9c-99f9-2ed706b24ea9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:25:54.615377 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78503550-91a8-4c9c-99f9-2ed706b24ea9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:25:54.616731 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78503550-91a8-4c9c-99f9-2ed706b24ea9 service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:25:54.635580 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 745/1487] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:25:54 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:25:56.818095 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19344bc-b09c-4706-8cc8-4911b1e7643a None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:56.818563 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19344bc-b09c-4706-8cc8-4911b1e7643a None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:56.818658 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19344bc-b09c-4706-8cc8-4911b1e7643a None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:56.820005 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 743/1488] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:25:56 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:25:56.826015 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67900219-d261-4c1b-ac15-925d08023cdc None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:56.826506 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67900219-d261-4c1b-ac15-925d08023cdc None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:56.826617 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67900219-d261-4c1b-ac15-925d08023cdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:56.892880 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-67900219-d261-4c1b-ac15-925d08023cdc None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:25:56.894123 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-67900219-d261-4c1b-ac15-925d08023cdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:25:56.950414 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:25:56.996093 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 746/1489] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:25:56 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:25:57.285624 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19344bc-b09c-4706-8cc8-4911b1e7643a None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:25:57.287183 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-c19344bc-b09c-4706-8cc8-4911b1e7643a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:25:57.323152 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:25:57.398496 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19344bc-b09c-4706-8cc8-4911b1e7643a None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:25:57.472151 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264aa631-2ddc-41f4-a596-c5a851186cfb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:25:57.472754 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264aa631-2ddc-41f4-a596-c5a851186cfb service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:57.473038 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264aa631-2ddc-41f4-a596-c5a851186cfb service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:57.473332 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264aa631-2ddc-41f4-a596-c5a851186cfb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:57.474953 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-264aa631-2ddc-41f4-a596-c5a851186cfb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:25:57.556606 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:25:57.590177 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264aa631-2ddc-41f4-a596-c5a851186cfb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:25:57.591864 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264aa631-2ddc-41f4-a596-c5a851186cfb service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:25:57.610485 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 744/1490] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:25:57 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:25:59.738591 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97f992c-e959-46c9-89b2-7367b1c485f1 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:59.739135 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97f992c-e959-46c9-89b2-7367b1c485f1 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:59.739135 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97f992c-e959-46c9-89b2-7367b1c485f1 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:59.740531 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 747/1491] 10.222.0.102 () {54 vars in 932 bytes} [Thu Sep 1 17:25:59 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:25:59.746864 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ace69b2-4a27-4a63-8825-a288b8caf5b3 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:59.747302 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ace69b2-4a27-4a63-8825-a288b8caf5b3 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:59.747410 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ace69b2-4a27-4a63-8825-a288b8caf5b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:59.813765 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-7ace69b2-4a27-4a63-8825-a288b8caf5b3 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:25:59.815049 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-7ace69b2-4a27-4a63-8825-a288b8caf5b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:25:59.912084 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 745/1492] 10.222.0.102 () {58 vars in 1029 bytes} [Thu Sep 1 17:25:59 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:26:00.198239 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97f992c-e959-46c9-89b2-7367b1c485f1 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:26:00.199883 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-f97f992c-e959-46c9-89b2-7367b1c485f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:26:00.305106 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97f992c-e959-46c9-89b2-7367b1c485f1 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:26:00.377455 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3878d617-cf59-4c09-9620-0dd7fc364c86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:26:00.378064 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3878d617-cf59-4c09-9620-0dd7fc364c86 service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:26:00.378344 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3878d617-cf59-4c09-9620-0dd7fc364c86 service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:26:00.378620 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3878d617-cf59-4c09-9620-0dd7fc364c86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:26:00.380239 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-3878d617-cf59-4c09-9620-0dd7fc364c86 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:26:00.390159 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:26:00.501830 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3878d617-cf59-4c09-9620-0dd7fc364c86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:26:00.503173 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3878d617-cf59-4c09-9620-0dd7fc364c86 service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:26:00.522211 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 748/1493] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:26:00 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 17:26:09.910266 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=82 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:26:10.519529 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=50 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:27:13.051890 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1114725e-0fa4-4e11-9eff-ffa07acfe625 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:27:13.052473 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1114725e-0fa4-4e11-9eff-ffa07acfe625 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:27:13.052473 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1114725e-0fa4-4e11-9eff-ffa07acfe625 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:27:13.132081 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-1114725e-0fa4-4e11-9eff-ffa07acfe625 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:27:13.133218 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-1114725e-0fa4-4e11-9eff-ffa07acfe625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:27:13.231979 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 746/1494] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:27:13 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:27:13.241401 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f90de9b-7f84-4de8-8e38-2f90653923b5 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:27:13.241898 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f90de9b-7f84-4de8-8e38-2f90653923b5 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:27:13.242006 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f90de9b-7f84-4de8-8e38-2f90653923b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:27:13.319746 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-7f90de9b-7f84-4de8-8e38-2f90653923b5 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:27:13.320960 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-7f90de9b-7f84-4de8-8e38-2f90653923b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:27:13.416813 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 749/1495] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:27:13 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:27:13.421743 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1114725e-0fa4-4e11-9eff-ffa07acfe625 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:27:13.422953 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-1114725e-0fa4-4e11-9eff-ffa07acfe625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:27:13.434950 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1114725e-0fa4-4e11-9eff-ffa07acfe625 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:27:13.494575 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:27:13.531579 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e1a916-b95e-4964-a4f0-87153bffd949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:27:13.532184 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e1a916-b95e-4964-a4f0-87153bffd949 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:27:13.532466 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e1a916-b95e-4964-a4f0-87153bffd949 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:27:13.532737 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e1a916-b95e-4964-a4f0-87153bffd949 admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:27:13.533472 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e1a916-b95e-4964-a4f0-87153bffd949 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:27:13.533780 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e1a916-b95e-4964-a4f0-87153bffd949 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:27:13.534879 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: WARNING py.warnings [None req-84e1a916-b95e-4964-a4f0-87153bffd949 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 01 17:27:13.534879 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: warnings.warn(msg) Sep 01 17:27:13.534879 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: Sep 01 17:27:13.536497 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e1a916-b95e-4964-a4f0-87153bffd949 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:27:13.549329 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 747/1496] 10.222.0.102 () {58 vars in 1158 bytes} [Thu Sep 1 17:27:13 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:27:13.557749 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f90de9b-7f84-4de8-8e38-2f90653923b5 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:27:13.566876 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f90de9b-7f84-4de8-8e38-2f90653923b5 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:27:13.627869 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:27:13.661202 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8428e1-6f53-4af7-b53a-e8d685531e41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:27:13.661899 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8428e1-6f53-4af7-b53a-e8d685531e41 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:27:13.662177 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8428e1-6f53-4af7-b53a-e8d685531e41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:27:13.662449 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8428e1-6f53-4af7-b53a-e8d685531e41 admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:27:13.663068 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8428e1-6f53-4af7-b53a-e8d685531e41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:27:13.664774 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: WARNING py.warnings [None req-ee8428e1-6f53-4af7-b53a-e8d685531e41 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 17:27:13.664774 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: warnings.warn(msg) Sep 01 17:27:13.664774 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: Sep 01 17:27:13.666007 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8428e1-6f53-4af7-b53a-e8d685531e41 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:27:13.721382 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 750/1497] 10.222.0.102 () {60 vars in 1158 bytes} [Thu Sep 1 17:27:13 2022] POST /identity/v3/projects => generated 378 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:27:13.725574 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e1a916-b95e-4964-a4f0-87153bffd949 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:27:13.727104 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-84e1a916-b95e-4964-a4f0-87153bffd949 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:27:13.760608 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e1a916-b95e-4964-a4f0-87153bffd949 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:27:13.854098 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48593419-25bc-4598-a9f1-c1d22ed159c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:27:13.854708 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48593419-25bc-4598-a9f1-c1d22ed159c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:27:13.854987 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48593419-25bc-4598-a9f1-c1d22ed159c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:27:13.855828 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48593419-25bc-4598-a9f1-c1d22ed159c9 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:27:13.856440 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48593419-25bc-4598-a9f1-c1d22ed159c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:27:13.857484 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: WARNING py.warnings [None req-48593419-25bc-4598-a9f1-c1d22ed159c9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 17:27:13.857484 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: warnings.warn(msg) Sep 01 17:27:13.857484 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: Sep 01 17:27:13.858715 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48593419-25bc-4598-a9f1-c1d22ed159c9 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:27:13.904091 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 748/1498] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:27:13 2022] POST /identity/v3/users => generated 290 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:27:13.908215 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8428e1-6f53-4af7-b53a-e8d685531e41 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:27:13.909772 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-ee8428e1-6f53-4af7-b53a-e8d685531e41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:27:13.971758 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8428e1-6f53-4af7-b53a-e8d685531e41 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:27:14.054550 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:27:14.067206 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506eccea-4786-42d0-92dd-77ccbe3770c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:27:14.067814 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506eccea-4786-42d0-92dd-77ccbe3770c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:27:14.068097 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506eccea-4786-42d0-92dd-77ccbe3770c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:27:14.068369 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506eccea-4786-42d0-92dd-77ccbe3770c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:27:14.068997 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506eccea-4786-42d0-92dd-77ccbe3770c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:27:14.070040 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: WARNING py.warnings [None req-506eccea-4786-42d0-92dd-77ccbe3770c7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 01 17:27:14.070040 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: warnings.warn(msg) Sep 01 17:27:14.070040 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: Sep 01 17:27:14.070943 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506eccea-4786-42d0-92dd-77ccbe3770c7 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:27:14.083421 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 751/1499] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:27:13 2022] GET /identity/v3/roles => generated 1427 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 17:27:14.087564 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48593419-25bc-4598-a9f1-c1d22ed159c9 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:27:14.089083 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-48593419-25bc-4598-a9f1-c1d22ed159c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:27:14.100690 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48593419-25bc-4598-a9f1-c1d22ed159c9 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:27:14.126781 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:27:14.196681 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7665350b-6aa6-496e-b3df-d7bd21a8f3dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:27:14.197296 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7665350b-6aa6-496e-b3df-d7bd21a8f3dc admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:27:14.197606 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7665350b-6aa6-496e-b3df-d7bd21a8f3dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:27:14.197881 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7665350b-6aa6-496e-b3df-d7bd21a8f3dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:27:14.198519 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7665350b-6aa6-496e-b3df-d7bd21a8f3dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:27:14.199583 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: WARNING py.warnings [None req-7665350b-6aa6-496e-b3df-d7bd21a8f3dc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 01 17:27:14.199583 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: warnings.warn(msg) Sep 01 17:27:14.199583 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: Sep 01 17:27:14.200480 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7665350b-6aa6-496e-b3df-d7bd21a8f3dc admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:27:14.212627 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 749/1500] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:27:14 2022] GET /identity/v3/roles => generated 1427 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:27:14.217035 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506eccea-4786-42d0-92dd-77ccbe3770c7 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:27:14.218628 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-506eccea-4786-42d0-92dd-77ccbe3770c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:27:14.254523 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506eccea-4786-42d0-92dd-77ccbe3770c7 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:27:14.350207 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a037775a-fad9-443e-a301-6e2c59f8f055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:27:14.350856 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a037775a-fad9-443e-a301-6e2c59f8f055 admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:27:14.351132 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a037775a-fad9-443e-a301-6e2c59f8f055 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:27:14.351402 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a037775a-fad9-443e-a301-6e2c59f8f055 admin admin] PATH_INFO: `/v3/projects/86f77161f3ea4cd48747b0fa4b6ef979/users/08ef861c936a420ea3a67cb84de06deb/roles/b75cb360925e4ba5a65f14965c103ffd` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:27:14.373806 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a037775a-fad9-443e-a301-6e2c59f8f055 admin admin] RBAC: Authorizing `identity:create_grant(project_id=86f77161f3ea4cd48747b0fa4b6ef979, user_id=08ef861c936a420ea3a67cb84de06deb, role_id=b75cb360925e4ba5a65f14965c103ffd)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:27:14.374821 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: WARNING py.warnings [None req-a037775a-fad9-443e-a301-6e2c59f8f055 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 17:27:14.374821 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: warnings.warn(msg) Sep 01 17:27:14.374821 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: Sep 01 17:27:14.376797 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a037775a-fad9-443e-a301-6e2c59f8f055 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:27:14.402938 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 752/1501] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:27:14 2022] PUT /identity/v3/projects/86f77161f3ea4cd48747b0fa4b6ef979/users/08ef861c936a420ea3a67cb84de06deb/roles/b75cb360925e4ba5a65f14965c103ffd => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:27:14.414947 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8bba12-7aa4-49fd-b50c-96ee73d7c408 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:27:14.415243 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8bba12-7aa4-49fd-b50c-96ee73d7c408 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:27:14.415527 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8bba12-7aa4-49fd-b50c-96ee73d7c408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:27:14.472941 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-0e8bba12-7aa4-49fd-b50c-96ee73d7c408 None None] MFA Rules not processed for user `08ef861c936a420ea3a67cb84de06deb`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:27:14.473862 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-0e8bba12-7aa4-49fd-b50c-96ee73d7c408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:27:14.552553 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 750/1502] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:27:14 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:27:14.565900 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d405b1d1-dfc9-44e8-8fce-7bfb228e0c5e None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:27:14.566451 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d405b1d1-dfc9-44e8-8fce-7bfb228e0c5e None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:27:14.566743 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d405b1d1-dfc9-44e8-8fce-7bfb228e0c5e None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:27:14.568090 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 753/1503] 10.222.0.102 () {54 vars in 969 bytes} [Thu Sep 1 17:27:14 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:27:14.573021 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91cca2e-434c-4ad5-8ff3-956373f9d0e2 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:27:14.573344 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91cca2e-434c-4ad5-8ff3-956373f9d0e2 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:27:14.573611 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91cca2e-434c-4ad5-8ff3-956373f9d0e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:27:14.604389 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:27:14.650170 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-c91cca2e-434c-4ad5-8ff3-956373f9d0e2 None None] MFA Rules not processed for user `523ed589872a45a09b6a7675a7fb0b75`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:27:14.651383 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-c91cca2e-434c-4ad5-8ff3-956373f9d0e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:27:14.726225 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 751/1504] 10.222.0.102 () {58 vars in 1066 bytes} [Thu Sep 1 17:27:14 2022] POST /identity/v3/auth/tokens => generated 3130 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:27:14.735883 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d405b1d1-dfc9-44e8-8fce-7bfb228e0c5e None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:27:14.737510 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-d405b1d1-dfc9-44e8-8fce-7bfb228e0c5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:27:14.792271 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:27:14.854650 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d405b1d1-dfc9-44e8-8fce-7bfb228e0c5e None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:27:14.928169 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42275d5-49c6-4dbc-8635-aba5db731d09 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '523ed589872a45a09b6a7675a7fb0b75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:27:14.928771 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42275d5-49c6-4dbc-8635-aba5db731d09 service cinder] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:27:14.929050 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42275d5-49c6-4dbc-8635-aba5db731d09 service cinder] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:27:14.929341 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42275d5-49c6-4dbc-8635-aba5db731d09 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:27:14.930950 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-b42275d5-49c6-4dbc-8635-aba5db731d09 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:27:15.028586 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42275d5-49c6-4dbc-8635-aba5db731d09 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:27:15.030044 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42275d5-49c6-4dbc-8635-aba5db731d09 service cinder] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:27:15.033464 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:27:15.048238 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 754/1505] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:27:14 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:27:15.093854 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7618ddc-acde-4357-8679-3f4ca89e4030 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:27:15.094288 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7618ddc-acde-4357-8679-3f4ca89e4030 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:27:15.094411 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7618ddc-acde-4357-8679-3f4ca89e4030 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:27:15.095940 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 752/1506] 10.222.0.102 () {52 vars in 835 bytes} [Thu Sep 1 17:27:15 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:27:15.099643 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42275d5-49c6-4dbc-8635-aba5db731d09 service cinder] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:27:15.100862 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-b42275d5-49c6-4dbc-8635-aba5db731d09 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:27:15.200791 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42275d5-49c6-4dbc-8635-aba5db731d09 service cinder] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:27:15.293892 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b12e4d9-fb9c-47ea-b0cc-3a6038ea9090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:27:15.294520 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b12e4d9-fb9c-47ea-b0cc-3a6038ea9090 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:27:15.294790 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b12e4d9-fb9c-47ea-b0cc-3a6038ea9090 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:27:15.295061 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b12e4d9-fb9c-47ea-b0cc-3a6038ea9090 admin admin] PATH_INFO: `/v3/users/08ef861c936a420ea3a67cb84de06deb` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:27:15.307427 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b12e4d9-fb9c-47ea-b0cc-3a6038ea9090 admin admin] RBAC: Authorizing `identity:delete_user(user_id=08ef861c936a420ea3a67cb84de06deb)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:27:15.308948 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: WARNING py.warnings [None req-4b12e4d9-fb9c-47ea-b0cc-3a6038ea9090 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 17:27:15.308948 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: warnings.warn(msg) Sep 01 17:27:15.308948 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: Sep 01 17:27:15.310198 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b12e4d9-fb9c-47ea-b0cc-3a6038ea9090 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:27:15.321918 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:27:15.379568 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-4b12e4d9-fb9c-47ea-b0cc-3a6038ea9090 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08ef861c936a420ea3a67cb84de06deb'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:27:15.380108 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-4b12e4d9-fb9c-47ea-b0cc-3a6038ea9090 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08ef861c936a420ea3a67cb84de06deb'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:27:15.402822 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-4b12e4d9-fb9c-47ea-b0cc-3a6038ea9090 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08ef861c936a420ea3a67cb84de06deb'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:27:15.411550 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-4b12e4d9-fb9c-47ea-b0cc-3a6038ea9090 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08ef861c936a420ea3a67cb84de06deb'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:27:15.412149 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-4b12e4d9-fb9c-47ea-b0cc-3a6038ea9090 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08ef861c936a420ea3a67cb84de06deb'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:27:15.431494 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 755/1507] 10.222.0.102 () {58 vars in 1229 bytes} [Thu Sep 1 17:27:15 2022] DELETE /identity/v3/users/08ef861c936a420ea3a67cb84de06deb => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:27:15.436487 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe97564-d301-4321-a1df-cfedf5cae3b4 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:27:15.436778 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe97564-d301-4321-a1df-cfedf5cae3b4 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:27:15.437053 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe97564-d301-4321-a1df-cfedf5cae3b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:27:15.493244 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-7fe97564-d301-4321-a1df-cfedf5cae3b4 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:27:15.494497 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-7fe97564-d301-4321-a1df-cfedf5cae3b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:27:15.570110 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:27:15.584110 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 753/1508] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:27:15 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:27:15.593612 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b12e4d9-fb9c-47ea-b0cc-3a6038ea9090 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:27:15.594846 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-4b12e4d9-fb9c-47ea-b0cc-3a6038ea9090 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:27:15.695161 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b12e4d9-fb9c-47ea-b0cc-3a6038ea9090 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:27:15.751597 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:27:15.771693 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d064b831-73d6-4b0a-9bcc-99c1630cc8ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:27:15.772296 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d064b831-73d6-4b0a-9bcc-99c1630cc8ee service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:27:15.772572 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d064b831-73d6-4b0a-9bcc-99c1630cc8ee service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:27:15.772842 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d064b831-73d6-4b0a-9bcc-99c1630cc8ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:27:15.774460 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-d064b831-73d6-4b0a-9bcc-99c1630cc8ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:27:15.884552 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d064b831-73d6-4b0a-9bcc-99c1630cc8ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:27:15.885944 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d064b831-73d6-4b0a-9bcc-99c1630cc8ee service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:27:15.906513 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 756/1509] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:27:15 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:27:16.257544 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe97564-d301-4321-a1df-cfedf5cae3b4 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:27:16.258895 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-7fe97564-d301-4321-a1df-cfedf5cae3b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:27:16.270845 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe97564-d301-4321-a1df-cfedf5cae3b4 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:27:16.368862 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa4c55c-ec53-4ac4-8e6c-4adf203719f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:27:16.369519 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa4c55c-ec53-4ac4-8e6c-4adf203719f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:27:16.369796 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa4c55c-ec53-4ac4-8e6c-4adf203719f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:27:16.370063 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa4c55c-ec53-4ac4-8e6c-4adf203719f5 admin admin] PATH_INFO: `/v3/projects/86f77161f3ea4cd48747b0fa4b6ef979` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:27:16.379835 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa4c55c-ec53-4ac4-8e6c-4adf203719f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=86f77161f3ea4cd48747b0fa4b6ef979)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:27:16.380849 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: WARNING py.warnings [None req-ffa4c55c-ec53-4ac4-8e6c-4adf203719f5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 17:27:16.380849 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: warnings.warn(msg) Sep 01 17:27:16.380849 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: Sep 01 17:27:16.382083 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa4c55c-ec53-4ac4-8e6c-4adf203719f5 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:27:16.442068 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-ffa4c55c-ec53-4ac4-8e6c-4adf203719f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86f77161f3ea4cd48747b0fa4b6ef979', 'request_id': 'req-ffa4c55c-ec53-4ac4-8e6c-4adf203719f5'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:27:16.452106 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-ffa4c55c-ec53-4ac4-8e6c-4adf203719f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86f77161f3ea4cd48747b0fa4b6ef979', 'request_id': 'req-ffa4c55c-ec53-4ac4-8e6c-4adf203719f5'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:27:16.467579 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-ffa4c55c-ec53-4ac4-8e6c-4adf203719f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86f77161f3ea4cd48747b0fa4b6ef979', 'request_id': 'req-ffa4c55c-ec53-4ac4-8e6c-4adf203719f5'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:27:16.489731 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-ffa4c55c-ec53-4ac4-8e6c-4adf203719f5 admin admin] The token cache is being invalidate because project 86f77161f3ea4cd48747b0fa4b6ef979 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:27:16.490018 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-ffa4c55c-ec53-4ac4-8e6c-4adf203719f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:27:16.490543 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-ffa4c55c-ec53-4ac4-8e6c-4adf203719f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:27:16.491848 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 754/1510] 10.222.0.102 () {58 vars in 1238 bytes} [Thu Sep 1 17:27:16 2022] DELETE /identity/v3/projects/86f77161f3ea4cd48747b0fa4b6ef979 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:27:25.903414 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=62 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:27:26.487709 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:28:51.862919 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d6e0c8-4289-4a80-80ba-7843da36f2b0 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:28:51.863479 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d6e0c8-4289-4a80-80ba-7843da36f2b0 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:28:51.863479 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d6e0c8-4289-4a80-80ba-7843da36f2b0 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:28:51.864931 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 757/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 17:28:51 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:28:51.869928 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb517eb-eb3f-476c-966b-58d01346c53a None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:28:51.870454 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb517eb-eb3f-476c-966b-58d01346c53a None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:28:51.870454 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb517eb-eb3f-476c-966b-58d01346c53a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:28:51.951177 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-3fb517eb-eb3f-476c-966b-58d01346c53a None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:28:51.952397 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-3fb517eb-eb3f-476c-966b-58d01346c53a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:28:52.057354 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 755/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 17:28:51 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:28:52.210255 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d6e0c8-4289-4a80-80ba-7843da36f2b0 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:28:52.212367 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-d2d6e0c8-4289-4a80-80ba-7843da36f2b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:28:52.328516 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d6e0c8-4289-4a80-80ba-7843da36f2b0 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:28:52.334332 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2aef8e-3757-40ed-ae7a-8d612840f259 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f8293316baf4774b62fc11d55d50865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:28:52.334929 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2aef8e-3757-40ed-ae7a-8d612840f259 service nova] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:28:52.335205 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2aef8e-3757-40ed-ae7a-8d612840f259 service nova] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:28:52.335473 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2aef8e-3757-40ed-ae7a-8d612840f259 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:28:52.343336 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2aef8e-3757-40ed-ae7a-8d612840f259 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:28:52.344902 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2aef8e-3757-40ed-ae7a-8d612840f259 service nova] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:28:52.355773 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 758/1513] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:28:52 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:28:53.182889 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cc9f81-c4af-4cce-a30e-b429f36f39f7 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:28:53.183422 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cc9f81-c4af-4cce-a30e-b429f36f39f7 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:28:53.183422 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cc9f81-c4af-4cce-a30e-b429f36f39f7 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:28:53.184688 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 756/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 17:28:53 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:28:53.190022 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078229e9-a50c-455d-ac22-a7c3a48eac5b None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:28:53.190475 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078229e9-a50c-455d-ac22-a7c3a48eac5b None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:28:53.190597 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078229e9-a50c-455d-ac22-a7c3a48eac5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:28:53.210215 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-078229e9-a50c-455d-ac22-a7c3a48eac5b None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:28:53.211410 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-078229e9-a50c-455d-ac22-a7c3a48eac5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:28:53.219170 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 759/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 17:28:53 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:28:53.370875 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cc9f81-c4af-4cce-a30e-b429f36f39f7 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:28:53.385577 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cc9f81-c4af-4cce-a30e-b429f36f39f7 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:28:53.391563 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2502e332-cdb2-47a4-bf26-503dce069e31 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f8293316baf4774b62fc11d55d50865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:28:53.392752 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2502e332-cdb2-47a4-bf26-503dce069e31 service nova] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:28:53.393029 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2502e332-cdb2-47a4-bf26-503dce069e31 service nova] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:28:53.393300 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2502e332-cdb2-47a4-bf26-503dce069e31 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:28:53.401203 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2502e332-cdb2-47a4-bf26-503dce069e31 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:28:53.402805 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2502e332-cdb2-47a4-bf26-503dce069e31 service nova] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:28:53.412263 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 757/1516] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:28:53 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:28:56.414168 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29210a1a-9a12-4ebf-830a-db164e982933 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:28:56.414780 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29210a1a-9a12-4ebf-830a-db164e982933 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:28:56.414780 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29210a1a-9a12-4ebf-830a-db164e982933 None None] PATH_INFO: `/v3` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:28:56.415993 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 760/1517] 10.222.0.73 () {54 vars in 929 bytes} [Thu Sep 1 17:28:56 2022] GET /identity/v3 => generated 256 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:28:56.423996 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49a096e-accf-42d9-bcd3-9b4385802339 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:28:56.424453 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49a096e-accf-42d9-bcd3-9b4385802339 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:28:56.424560 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49a096e-accf-42d9-bcd3-9b4385802339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:28:56.485229 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-c49a096e-accf-42d9-bcd3-9b4385802339 None None] MFA Rules not processed for user `134ef6039cef468bae0e3f3be79b8073`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:28:56.486482 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-c49a096e-accf-42d9-bcd3-9b4385802339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:28:56.503939 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:28:56.564338 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 758/1518] 10.222.0.73 () {58 vars in 1019 bytes} [Thu Sep 1 17:28:56 2022] POST /identity/v3/auth/tokens => generated 3318 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:29:03.204949 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=56 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:29:04.633974 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a831e1a1-b35e-40b2-bfe5-2b7f0c9459e0 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:29:04.634660 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a831e1a1-b35e-40b2-bfe5-2b7f0c9459e0 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:29:04.634660 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a831e1a1-b35e-40b2-bfe5-2b7f0c9459e0 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:29:04.635888 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 761/1519] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 17:29:04 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:29:04.640822 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754ef636-41cd-4822-99a0-a9767c2b39d8 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:29:04.641275 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754ef636-41cd-4822-99a0-a9767c2b39d8 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:29:04.641464 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754ef636-41cd-4822-99a0-a9767c2b39d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:29:04.661194 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-754ef636-41cd-4822-99a0-a9767c2b39d8 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:29:04.662464 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-754ef636-41cd-4822-99a0-a9767c2b39d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:29:04.670284 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 759/1520] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 17:29:04 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:29:04.832154 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a831e1a1-b35e-40b2-bfe5-2b7f0c9459e0 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:29:04.847260 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a831e1a1-b35e-40b2-bfe5-2b7f0c9459e0 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:29:04.853074 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f297fb3-4435-4a1f-a4d7-53a1881c0b58 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f8293316baf4774b62fc11d55d50865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:29:04.853796 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f297fb3-4435-4a1f-a4d7-53a1881c0b58 service nova] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:29:04.854078 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f297fb3-4435-4a1f-a4d7-53a1881c0b58 service nova] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:29:04.854356 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f297fb3-4435-4a1f-a4d7-53a1881c0b58 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:29:04.863794 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f297fb3-4435-4a1f-a4d7-53a1881c0b58 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:29:04.865403 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f297fb3-4435-4a1f-a4d7-53a1881c0b58 service nova] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:29:04.875139 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 762/1521] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:29:04 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:29:05.689063 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0023b83-527a-4b6e-a39a-5d5e5829ce10 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:29:05.689677 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0023b83-527a-4b6e-a39a-5d5e5829ce10 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:29:05.689677 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0023b83-527a-4b6e-a39a-5d5e5829ce10 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:29:05.691017 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 760/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 17:29:05 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:29:05.695746 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fad492-4127-47fa-a623-b943af726eb0 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:29:05.696192 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fad492-4127-47fa-a623-b943af726eb0 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:29:05.696291 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fad492-4127-47fa-a623-b943af726eb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:29:05.715873 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-c3fad492-4127-47fa-a623-b943af726eb0 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:29:05.717221 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-c3fad492-4127-47fa-a623-b943af726eb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:29:05.725921 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 763/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 17:29:05 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:29:05.877410 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0023b83-527a-4b6e-a39a-5d5e5829ce10 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:29:05.891584 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0023b83-527a-4b6e-a39a-5d5e5829ce10 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:29:05.897563 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57500a3-07e4-469e-bf98-d962c511b1ca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f8293316baf4774b62fc11d55d50865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:29:05.898154 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57500a3-07e4-469e-bf98-d962c511b1ca service nova] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:29:05.898431 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57500a3-07e4-469e-bf98-d962c511b1ca service nova] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:29:05.898703 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57500a3-07e4-469e-bf98-d962c511b1ca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:29:05.906889 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57500a3-07e4-469e-bf98-d962c511b1ca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:29:05.908514 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57500a3-07e4-469e-bf98-d962c511b1ca service nova] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:29:05.917798 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 761/1524] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:29:05 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:29:08.471461 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5390484-ec03-4671-86db-77db6dc4014f None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:29:08.471996 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5390484-ec03-4671-86db-77db6dc4014f None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:29:08.471996 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5390484-ec03-4671-86db-77db6dc4014f None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:29:08.473597 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 764/1525] 10.222.0.102 () {54 vars in 966 bytes} [Thu Sep 1 17:29:08 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:29:08.478400 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcdd463-7ead-4088-ae89-d248426c1815 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:29:08.478811 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcdd463-7ead-4088-ae89-d248426c1815 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:29:08.478960 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcdd463-7ead-4088-ae89-d248426c1815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:29:08.497957 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-fdcdd463-7ead-4088-ae89-d248426c1815 None None] MFA Rules not processed for user `134ef6039cef468bae0e3f3be79b8073`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:29:08.499161 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-fdcdd463-7ead-4088-ae89-d248426c1815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:29:08.508233 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 762/1526] 10.222.0.102 () {58 vars in 1063 bytes} [Thu Sep 1 17:29:08 2022] POST /identity/v3/auth/tokens => generated 3318 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:29:08.515536 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5390484-ec03-4671-86db-77db6dc4014f None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:29:08.529761 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5390484-ec03-4671-86db-77db6dc4014f None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:29:08.534765 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df45644e-c908-4fbb-a879-74fe4fec47e9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '134ef6039cef468bae0e3f3be79b8073', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:29:08.535381 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df45644e-c908-4fbb-a879-74fe4fec47e9 service placement] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:29:08.535662 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df45644e-c908-4fbb-a879-74fe4fec47e9 service placement] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:29:08.535939 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df45644e-c908-4fbb-a879-74fe4fec47e9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:29:08.544095 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df45644e-c908-4fbb-a879-74fe4fec47e9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:29:08.545694 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df45644e-c908-4fbb-a879-74fe4fec47e9 service placement] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:29:08.555762 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 765/1527] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:29:08 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:29:17.200799 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd723b2-7fc5-4245-a600-7f73bd6c3c4f None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:29:17.201351 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd723b2-7fc5-4245-a600-7f73bd6c3c4f None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:29:17.201351 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd723b2-7fc5-4245-a600-7f73bd6c3c4f None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:29:17.202652 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 763/1528] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 17:29:17 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:29:17.208063 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0692d67-106d-4a2e-870d-2860ba484d57 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:29:17.208454 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0692d67-106d-4a2e-870d-2860ba484d57 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:29:17.208614 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0692d67-106d-4a2e-870d-2860ba484d57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:29:17.228214 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-c0692d67-106d-4a2e-870d-2860ba484d57 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:29:17.229478 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-c0692d67-106d-4a2e-870d-2860ba484d57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:29:17.237970 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 766/1529] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 17:29:17 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:29:17.397174 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd723b2-7fc5-4245-a600-7f73bd6c3c4f None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:29:17.411345 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd723b2-7fc5-4245-a600-7f73bd6c3c4f None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:29:17.416823 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba790070-1495-4f68-bc5b-340e1c14276b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f8293316baf4774b62fc11d55d50865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:29:17.417457 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba790070-1495-4f68-bc5b-340e1c14276b service nova] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:29:17.417740 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba790070-1495-4f68-bc5b-340e1c14276b service nova] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:29:17.418017 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba790070-1495-4f68-bc5b-340e1c14276b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:29:17.426006 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba790070-1495-4f68-bc5b-340e1c14276b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:29:17.427595 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba790070-1495-4f68-bc5b-340e1c14276b service nova] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:29:17.436821 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 764/1530] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:29:17 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:29:18.254345 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc262461-3fc5-45a1-893a-8424533bf452 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:29:18.254906 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc262461-3fc5-45a1-893a-8424533bf452 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:29:18.254906 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc262461-3fc5-45a1-893a-8424533bf452 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:29:18.256187 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 767/1531] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 17:29:18 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:29:18.260912 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd530de6-95f3-430e-88d9-df7c5fd7a554 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:29:18.261365 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd530de6-95f3-430e-88d9-df7c5fd7a554 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:29:18.261494 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd530de6-95f3-430e-88d9-df7c5fd7a554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:29:18.282367 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-cd530de6-95f3-430e-88d9-df7c5fd7a554 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:29:18.283573 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-cd530de6-95f3-430e-88d9-df7c5fd7a554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:29:18.294961 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 765/1532] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 17:29:18 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:29:18.455268 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc262461-3fc5-45a1-893a-8424533bf452 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:29:18.469889 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc262461-3fc5-45a1-893a-8424533bf452 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:29:18.476326 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f326789c-3aae-4ed7-a7df-5bed213ab90b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f8293316baf4774b62fc11d55d50865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:29:18.476919 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f326789c-3aae-4ed7-a7df-5bed213ab90b service nova] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:29:18.477201 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f326789c-3aae-4ed7-a7df-5bed213ab90b service nova] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:29:18.477506 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f326789c-3aae-4ed7-a7df-5bed213ab90b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:29:18.485529 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f326789c-3aae-4ed7-a7df-5bed213ab90b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:29:18.487140 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f326789c-3aae-4ed7-a7df-5bed213ab90b service nova] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:29:18.496529 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 768/1533] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:29:18 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:29:22.285732 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4ef8cf-cbda-4423-8568-27cbab212d03 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:29:22.286258 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4ef8cf-cbda-4423-8568-27cbab212d03 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:29:22.286258 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4ef8cf-cbda-4423-8568-27cbab212d03 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:29:22.287537 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 766/1534] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 17:29:22 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:29:22.292562 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8caa78-bf60-455b-99ec-0f816c2e7485 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:29:22.292951 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8caa78-bf60-455b-99ec-0f816c2e7485 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:29:22.293120 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8caa78-bf60-455b-99ec-0f816c2e7485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:29:22.312807 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-9c8caa78-bf60-455b-99ec-0f816c2e7485 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:29:22.314038 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-9c8caa78-bf60-455b-99ec-0f816c2e7485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:29:22.326497 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 769/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 17:29:22 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:29:22.626967 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4ef8cf-cbda-4423-8568-27cbab212d03 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:29:22.628665 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-6e4ef8cf-cbda-4423-8568-27cbab212d03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:29:22.709263 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4ef8cf-cbda-4423-8568-27cbab212d03 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:29:22.712989 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cd0911-fc42-4cd0-be85-2fecdb145706 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:29:22.713634 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cd0911-fc42-4cd0-be85-2fecdb145706 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:29:22.713916 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cd0911-fc42-4cd0-be85-2fecdb145706 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:29:22.714190 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cd0911-fc42-4cd0-be85-2fecdb145706 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:29:22.723902 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cd0911-fc42-4cd0-be85-2fecdb145706 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:29:22.725355 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cd0911-fc42-4cd0-be85-2fecdb145706 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:29:22.734596 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 767/1536] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:29:22 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:29:24.631644 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8002631-2abe-4834-9b32-6fbbb2b44a69 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:29:24.632193 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8002631-2abe-4834-9b32-6fbbb2b44a69 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:29:24.632193 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8002631-2abe-4834-9b32-6fbbb2b44a69 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:29:24.633559 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 770/1537] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 17:29:24 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:29:24.638799 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee014b3-ba8d-4c3c-b694-6aae4ee60e56 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:29:24.639233 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee014b3-ba8d-4c3c-b694-6aae4ee60e56 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:29:24.639338 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee014b3-ba8d-4c3c-b694-6aae4ee60e56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:29:24.659481 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-2ee014b3-ba8d-4c3c-b694-6aae4ee60e56 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:29:24.660686 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-2ee014b3-ba8d-4c3c-b694-6aae4ee60e56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:29:24.668517 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 768/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 17:29:24 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:29:24.958566 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8002631-2abe-4834-9b32-6fbbb2b44a69 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:29:24.960232 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-c8002631-2abe-4834-9b32-6fbbb2b44a69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:29:24.974038 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8002631-2abe-4834-9b32-6fbbb2b44a69 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:29:24.977480 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cbc7f7-7608-4b23-b63a-45de652b38ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:29:24.978125 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cbc7f7-7608-4b23-b63a-45de652b38ae service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:29:24.978402 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cbc7f7-7608-4b23-b63a-45de652b38ae service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:29:24.978667 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cbc7f7-7608-4b23-b63a-45de652b38ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:29:24.988427 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cbc7f7-7608-4b23-b63a-45de652b38ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:29:24.989878 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cbc7f7-7608-4b23-b63a-45de652b38ae service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:29:24.999202 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 771/1539] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:29:24 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:29:28.946516 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df4f9be-3f5f-4781-bb02-0439471dd626 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:29:28.947100 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df4f9be-3f5f-4781-bb02-0439471dd626 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:29:28.947100 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df4f9be-3f5f-4781-bb02-0439471dd626 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:29:28.948401 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 769/1540] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 17:29:28 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:29:28.953609 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb0b104-eb0e-43f8-99c7-59179369405f None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:29:28.954031 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb0b104-eb0e-43f8-99c7-59179369405f None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:29:28.954165 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb0b104-eb0e-43f8-99c7-59179369405f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:29:28.974679 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-afb0b104-eb0e-43f8-99c7-59179369405f None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:29:28.976161 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-afb0b104-eb0e-43f8-99c7-59179369405f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:29:28.986052 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 772/1541] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 17:29:28 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:29:29.546048 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df4f9be-3f5f-4781-bb02-0439471dd626 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:29:29.547724 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-4df4f9be-3f5f-4781-bb02-0439471dd626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:29:29.552339 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=83 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:29:29.626363 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df4f9be-3f5f-4781-bb02-0439471dd626 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:29:29.630540 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e8c8a7-b603-4e08-83b2-f0709b0533c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '523ed589872a45a09b6a7675a7fb0b75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:29:29.631136 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e8c8a7-b603-4e08-83b2-f0709b0533c1 service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:29:29.631412 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e8c8a7-b603-4e08-83b2-f0709b0533c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:29:29.631686 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e8c8a7-b603-4e08-83b2-f0709b0533c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:29:29.640433 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e8c8a7-b603-4e08-83b2-f0709b0533c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:29:29.641897 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e8c8a7-b603-4e08-83b2-f0709b0533c1 service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:29:29.651091 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 770/1542] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:29:29 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:29:31.572770 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302bfea6-1a0c-4890-bf81-66191c775c91 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:29:31.573439 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302bfea6-1a0c-4890-bf81-66191c775c91 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:29:31.573559 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302bfea6-1a0c-4890-bf81-66191c775c91 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:29:31.574868 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 773/1543] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 17:29:31 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:29:31.579992 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9817f5ae-21e0-4c1d-b69e-01de6d2c615b None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:29:31.580457 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9817f5ae-21e0-4c1d-b69e-01de6d2c615b None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:29:31.580564 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9817f5ae-21e0-4c1d-b69e-01de6d2c615b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:29:31.600665 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-9817f5ae-21e0-4c1d-b69e-01de6d2c615b None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:29:31.601894 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-9817f5ae-21e0-4c1d-b69e-01de6d2c615b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:29:31.609857 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 771/1544] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 17:29:31 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:29:32.162922 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302bfea6-1a0c-4890-bf81-66191c775c91 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:29:32.164612 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-302bfea6-1a0c-4890-bf81-66191c775c91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:29:32.177286 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302bfea6-1a0c-4890-bf81-66191c775c91 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:29:32.180736 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3b8359-60b3-4c26-8d12-8fe852b0aa7d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '523ed589872a45a09b6a7675a7fb0b75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:29:32.181332 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3b8359-60b3-4c26-8d12-8fe852b0aa7d service cinder] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:29:32.181618 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3b8359-60b3-4c26-8d12-8fe852b0aa7d service cinder] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:29:32.181884 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3b8359-60b3-4c26-8d12-8fe852b0aa7d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:29:32.191027 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3b8359-60b3-4c26-8d12-8fe852b0aa7d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:29:32.192437 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3b8359-60b3-4c26-8d12-8fe852b0aa7d service cinder] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:29:32.201724 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 774/1545] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:29:32 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:29:41.594999 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=37 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:29:42.197526 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=42 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:05.255550 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8227f9d3-9297-4fa2-b5c5-9c59a59cec9d None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:05.256219 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8227f9d3-9297-4fa2-b5c5-9c59a59cec9d None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:05.256219 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8227f9d3-9297-4fa2-b5c5-9c59a59cec9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:05.281614 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-8227f9d3-9297-4fa2-b5c5-9c59a59cec9d None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:05.282827 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-8227f9d3-9297-4fa2-b5c5-9c59a59cec9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:05.297863 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4d9bef-0d81-4de5-bb04-01e8356ce987 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:05.298747 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 772/1546] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:05 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:05.298880 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4d9bef-0d81-4de5-bb04-01e8356ce987 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:05.298880 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4d9bef-0d81-4de5-bb04-01e8356ce987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:05.307210 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b68236-e372-4c38-a73c-894a55c6d78a None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:05.307580 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b68236-e372-4c38-a73c-894a55c6d78a None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:05.307782 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b68236-e372-4c38-a73c-894a55c6d78a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:05.336528 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-bd4d9bef-0d81-4de5-bb04-01e8356ce987 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:05.337827 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-bd4d9bef-0d81-4de5-bb04-01e8356ce987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:05.341788 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-85b68236-e372-4c38-a73c-894a55c6d78a None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:05.343008 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-85b68236-e372-4c38-a73c-894a55c6d78a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:05.359922 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 773/1547] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:05 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:05.361889 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b68236-e372-4c38-a73c-894a55c6d78a None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:05.363641 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 775/1548] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:05 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:05.374097 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26d844e-064c-4189-8368-c2dd5efdb801 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:05.375070 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26d844e-064c-4189-8368-c2dd5efdb801 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:05.375196 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26d844e-064c-4189-8368-c2dd5efdb801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:05.375591 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b68236-e372-4c38-a73c-894a55c6d78a None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:05.381114 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8db4a64-e52d-474c-9ee7-e7c2e1b98663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:05.381809 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8db4a64-e52d-474c-9ee7-e7c2e1b98663 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:05.382168 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8db4a64-e52d-474c-9ee7-e7c2e1b98663 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:05.382439 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8db4a64-e52d-474c-9ee7-e7c2e1b98663 admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:05.383169 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8db4a64-e52d-474c-9ee7-e7c2e1b98663 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:31:05.383479 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8db4a64-e52d-474c-9ee7-e7c2e1b98663 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:05.385087 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8db4a64-e52d-474c-9ee7-e7c2e1b98663 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:05.406186 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 774/1549] 10.222.0.102 () {58 vars in 1158 bytes} [Thu Sep 1 17:31:05 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:31:05.414360 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8db4a64-e52d-474c-9ee7-e7c2e1b98663 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:05.418839 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-f26d844e-064c-4189-8368-c2dd5efdb801 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:05.420081 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-f26d844e-064c-4189-8368-c2dd5efdb801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:05.432362 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8db4a64-e52d-474c-9ee7-e7c2e1b98663 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:05.440940 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f775b14d-1228-4662-99ae-4f83e4504c16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:05.441583 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f775b14d-1228-4662-99ae-4f83e4504c16 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:05.441862 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f775b14d-1228-4662-99ae-4f83e4504c16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:05.442137 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f775b14d-1228-4662-99ae-4f83e4504c16 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:05.442973 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f775b14d-1228-4662-99ae-4f83e4504c16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:05.444007 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: WARNING py.warnings [None req-f775b14d-1228-4662-99ae-4f83e4504c16 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 17:31:05.444007 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: warnings.warn(msg) Sep 01 17:31:05.444007 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: Sep 01 17:31:05.445076 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 776/1550] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:05 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:05.446323 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f775b14d-1228-4662-99ae-4f83e4504c16 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:05.450647 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26d844e-064c-4189-8368-c2dd5efdb801 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:05.465272 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26d844e-064c-4189-8368-c2dd5efdb801 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:05.490074 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd502f7-109b-4c70-b0de-9b87ccc943a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:05.490705 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd502f7-109b-4c70-b0de-9b87ccc943a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:05.490987 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd502f7-109b-4c70-b0de-9b87ccc943a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:05.491257 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd502f7-109b-4c70-b0de-9b87ccc943a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:05.492275 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd502f7-109b-4c70-b0de-9b87ccc943a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:31:05.492964 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd502f7-109b-4c70-b0de-9b87ccc943a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:05.493859 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: WARNING py.warnings [None req-9fd502f7-109b-4c70-b0de-9b87ccc943a5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 01 17:31:05.493859 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: warnings.warn(msg) Sep 01 17:31:05.493859 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: Sep 01 17:31:05.494773 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd502f7-109b-4c70-b0de-9b87ccc943a5 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:05.511947 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 777/1551] 10.222.0.102 () {58 vars in 1158 bytes} [Thu Sep 1 17:31:05 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:31:05.513008 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967dc5c1-cd2f-43c9-aa22-a1101b42a170 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:05.513299 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967dc5c1-cd2f-43c9-aa22-a1101b42a170 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:05.513597 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967dc5c1-cd2f-43c9-aa22-a1101b42a170 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:05.519942 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 775/1552] 10.222.0.102 () {60 vars in 1158 bytes} [Thu Sep 1 17:31:05 2022] POST /identity/v3/projects => generated 382 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:31:05.523334 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b16c1d1-0eeb-4d8f-bcee-32122622dcf0 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:05.523636 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b16c1d1-0eeb-4d8f-bcee-32122622dcf0 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:05.523906 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b16c1d1-0eeb-4d8f-bcee-32122622dcf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:05.553664 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-967dc5c1-cd2f-43c9-aa22-a1101b42a170 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:05.557213 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-967dc5c1-cd2f-43c9-aa22-a1101b42a170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:05.564136 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-4b16c1d1-0eeb-4d8f-bcee-32122622dcf0 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:05.569114 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-4b16c1d1-0eeb-4d8f-bcee-32122622dcf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:05.645110 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 778/1553] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:05 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:05.645730 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967dc5c1-cd2f-43c9-aa22-a1101b42a170 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:05.655277 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967dc5c1-cd2f-43c9-aa22-a1101b42a170 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:05.659977 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab7c25d-0c34-4ca7-9fbe-421bec332bf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:05.660720 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab7c25d-0c34-4ca7-9fbe-421bec332bf1 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:05.660997 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab7c25d-0c34-4ca7-9fbe-421bec332bf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:05.661269 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab7c25d-0c34-4ca7-9fbe-421bec332bf1 admin admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:05.662095 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab7c25d-0c34-4ca7-9fbe-421bec332bf1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:05.663102 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: WARNING py.warnings [None req-eab7c25d-0c34-4ca7-9fbe-421bec332bf1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 17:31:05.663102 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: warnings.warn(msg) Sep 01 17:31:05.663102 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: Sep 01 17:31:05.664300 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab7c25d-0c34-4ca7-9fbe-421bec332bf1 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:05.666860 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 776/1554] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:05 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:05.667696 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b16c1d1-0eeb-4d8f-bcee-32122622dcf0 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:05.679326 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b16c1d1-0eeb-4d8f-bcee-32122622dcf0 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:05.684442 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49645ac4-10ab-470a-8e5b-caa55f334ecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:05.685058 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49645ac4-10ab-470a-8e5b-caa55f334ecf admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:05.685365 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49645ac4-10ab-470a-8e5b-caa55f334ecf admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:05.685644 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49645ac4-10ab-470a-8e5b-caa55f334ecf admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:05.686281 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49645ac4-10ab-470a-8e5b-caa55f334ecf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:05.688440 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 779/1555] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:31:05 2022] POST /identity/v3/users => generated 292 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:31:05.688769 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49645ac4-10ab-470a-8e5b-caa55f334ecf admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:05.692640 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc8a612-7d25-4711-9276-8a80b47da093 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:05.692640 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc8a612-7d25-4711-9276-8a80b47da093 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:05.692640 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc8a612-7d25-4711-9276-8a80b47da093 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:05.710829 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-7fc8a612-7d25-4711-9276-8a80b47da093 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:05.712138 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-7fc8a612-7d25-4711-9276-8a80b47da093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:05.725179 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 777/1556] 10.222.0.102 () {60 vars in 1158 bytes} [Thu Sep 1 17:31:05 2022] POST /identity/v3/projects => generated 392 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:31:05.725400 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 780/1557] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:05 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:05.725400 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3e3fe3-e23c-49ac-b09d-3d10356d8960 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:05.725400 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3e3fe3-e23c-49ac-b09d-3d10356d8960 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:05.725400 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3e3fe3-e23c-49ac-b09d-3d10356d8960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:05.730503 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28782a3b-eb92-4bbd-bee3-03844e0e6cc3 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:05.730789 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28782a3b-eb92-4bbd-bee3-03844e0e6cc3 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:05.731324 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28782a3b-eb92-4bbd-bee3-03844e0e6cc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:05.745770 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-0d3e3fe3-e23c-49ac-b09d-3d10356d8960 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:05.746994 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-0d3e3fe3-e23c-49ac-b09d-3d10356d8960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:05.751667 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-28782a3b-eb92-4bbd-bee3-03844e0e6cc3 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:05.752845 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-28782a3b-eb92-4bbd-bee3-03844e0e6cc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:05.806558 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 781/1558] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:05 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:05.807833 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faaabdd-227a-4a19-9c02-727e44d57556 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:05.808119 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faaabdd-227a-4a19-9c02-727e44d57556 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:05.808392 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faaabdd-227a-4a19-9c02-727e44d57556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:05.820583 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 778/1559] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:05 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:05.822821 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19cef1f-48a5-4d75-90c7-73f60324134c None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:05.822821 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19cef1f-48a5-4d75-90c7-73f60324134c None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:05.822821 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19cef1f-48a5-4d75-90c7-73f60324134c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:05.834424 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-4faaabdd-227a-4a19-9c02-727e44d57556 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:05.835659 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-4faaabdd-227a-4a19-9c02-727e44d57556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:05.841882 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-d19cef1f-48a5-4d75-90c7-73f60324134c None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:05.843048 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-d19cef1f-48a5-4d75-90c7-73f60324134c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:05.844005 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 782/1560] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:05 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:05.845515 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bb7567-3c44-416f-825a-fd6f4f4a5eb7 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:05.845515 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bb7567-3c44-416f-825a-fd6f4f4a5eb7 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:05.845836 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bb7567-3c44-416f-825a-fd6f4f4a5eb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:05.857583 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 779/1561] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:05 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:05.858426 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19cef1f-48a5-4d75-90c7-73f60324134c None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:05.868648 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19cef1f-48a5-4d75-90c7-73f60324134c None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:05.872443 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-61bb7567-3c44-416f-825a-fd6f4f4a5eb7 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:05.873347 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712a4394-2703-40d8-b0cc-4428dd5b1015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:05.873674 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-61bb7567-3c44-416f-825a-fd6f4f4a5eb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:05.873945 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712a4394-2703-40d8-b0cc-4428dd5b1015 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:05.874257 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712a4394-2703-40d8-b0cc-4428dd5b1015 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:05.874569 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712a4394-2703-40d8-b0cc-4428dd5b1015 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:05.875210 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712a4394-2703-40d8-b0cc-4428dd5b1015 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:05.876812 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712a4394-2703-40d8-b0cc-4428dd5b1015 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:05.881493 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 783/1562] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:05 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:05.885158 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bb7567-3c44-416f-825a-fd6f4f4a5eb7 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:05.886903 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 780/1563] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:31:05 2022] GET /identity/v3/roles => generated 1427 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:31:05.888415 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8316538-bb40-47ba-a19d-2cbe2192f78e None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:05.888705 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8316538-bb40-47ba-a19d-2cbe2192f78e None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:05.888983 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8316538-bb40-47ba-a19d-2cbe2192f78e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:05.892469 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bb7567-3c44-416f-825a-fd6f4f4a5eb7 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:05.896948 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604ada58-5e14-449f-9579-da9fc51fc775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:05.897460 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:05.897601 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604ada58-5e14-449f-9579-da9fc51fc775 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:05.897830 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604ada58-5e14-449f-9579-da9fc51fc775 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:05.898104 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604ada58-5e14-449f-9579-da9fc51fc775 admin admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:05.898724 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604ada58-5e14-449f-9579-da9fc51fc775 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:05.900834 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604ada58-5e14-449f-9579-da9fc51fc775 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:05.909737 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-f8316538-bb40-47ba-a19d-2cbe2192f78e None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:05.910945 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-f8316538-bb40-47ba-a19d-2cbe2192f78e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:05.919577 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 781/1564] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:05 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:05.920805 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b7dcd9-a248-45e5-bb21-92bdca5f88c1 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:05.921089 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b7dcd9-a248-45e5-bb21-92bdca5f88c1 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:05.921457 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b7dcd9-a248-45e5-bb21-92bdca5f88c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:05.927206 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 784/1565] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:31:05 2022] POST /identity/v3/users => generated 297 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:31:05.928649 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b0b6b3-ea71-4888-a473-95d5e4990d84 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:05.928935 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b0b6b3-ea71-4888-a473-95d5e4990d84 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:05.929207 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b0b6b3-ea71-4888-a473-95d5e4990d84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:05.943529 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-a7b7dcd9-a248-45e5-bb21-92bdca5f88c1 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:05.944721 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-a7b7dcd9-a248-45e5-bb21-92bdca5f88c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:05.948400 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-e2b0b6b3-ea71-4888-a473-95d5e4990d84 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:05.949669 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-e2b0b6b3-ea71-4888-a473-95d5e4990d84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:05.953577 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 782/1566] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:05 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:05.955374 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc3afcd-f099-4c00-b3a2-7348d40b4531 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:05.955795 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc3afcd-f099-4c00-b3a2-7348d40b4531 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:05.956208 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc3afcd-f099-4c00-b3a2-7348d40b4531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:05.958930 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 785/1567] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:05 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:05.959727 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b0b6b3-ea71-4888-a473-95d5e4990d84 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:05.969934 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b0b6b3-ea71-4888-a473-95d5e4990d84 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:05.974739 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb646d7-b23d-4010-8c8c-c8cc98315b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:05.975320 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb646d7-b23d-4010-8c8c-c8cc98315b35 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:05.975600 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb646d7-b23d-4010-8c8c-c8cc98315b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:05.975871 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb646d7-b23d-4010-8c8c-c8cc98315b35 admin admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:05.976517 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb646d7-b23d-4010-8c8c-c8cc98315b35 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:31:05.976820 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb646d7-b23d-4010-8c8c-c8cc98315b35 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:05.978596 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb646d7-b23d-4010-8c8c-c8cc98315b35 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:05.987896 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:05.988076 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-7cc3afcd-f099-4c00-b3a2-7348d40b4531 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:05.989454 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-7cc3afcd-f099-4c00-b3a2-7348d40b4531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:05.993502 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 786/1568] 10.222.0.102 () {58 vars in 1158 bytes} [Thu Sep 1 17:31:05 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:31:05.999783 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb646d7-b23d-4010-8c8c-c8cc98315b35 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:05.999901 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 783/1569] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:05 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:06.000000 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc3afcd-f099-4c00-b3a2-7348d40b4531 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:06.009782 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc3afcd-f099-4c00-b3a2-7348d40b4531 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:06.010081 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb646d7-b23d-4010-8c8c-c8cc98315b35 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:06.014617 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f86ac7-0b1f-40eb-8b70-5ccc4e64ad9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:06.015268 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f86ac7-0b1f-40eb-8b70-5ccc4e64ad9c admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:06.015544 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f86ac7-0b1f-40eb-8b70-5ccc4e64ad9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:06.015768 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7e7f1d-1fd4-4817-a8bb-a910af91d1b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:06.015905 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f86ac7-0b1f-40eb-8b70-5ccc4e64ad9c admin admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:06.017046 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f86ac7-0b1f-40eb-8b70-5ccc4e64ad9c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:31:06.017145 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7e7f1d-1fd4-4817-a8bb-a910af91d1b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:06.017244 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f86ac7-0b1f-40eb-8b70-5ccc4e64ad9c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:06.017371 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7e7f1d-1fd4-4817-a8bb-a910af91d1b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:06.017371 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7e7f1d-1fd4-4817-a8bb-a910af91d1b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:06.018048 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7e7f1d-1fd4-4817-a8bb-a910af91d1b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:06.019474 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f86ac7-0b1f-40eb-8b70-5ccc4e64ad9c admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:06.019790 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7e7f1d-1fd4-4817-a8bb-a910af91d1b0 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:06.030518 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 784/1570] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:31:05 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:31:06.031829 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7e7f1d-1fd4-4817-a8bb-a910af91d1b0 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:06.040322 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 787/1571] 10.222.0.102 () {58 vars in 1158 bytes} [Thu Sep 1 17:31:05 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:31:06.041705 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f86ac7-0b1f-40eb-8b70-5ccc4e64ad9c admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:06.042233 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7e7f1d-1fd4-4817-a8bb-a910af91d1b0 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:06.048249 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9a449c-e62a-4699-9b3c-75466e42af74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:06.048859 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9a449c-e62a-4699-9b3c-75466e42af74 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:06.049146 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9a449c-e62a-4699-9b3c-75466e42af74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:06.050114 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9a449c-e62a-4699-9b3c-75466e42af74 admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:06.050114 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9a449c-e62a-4699-9b3c-75466e42af74 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:31:06.050943 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9a449c-e62a-4699-9b3c-75466e42af74 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:06.051991 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9a449c-e62a-4699-9b3c-75466e42af74 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:06.057984 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f86ac7-0b1f-40eb-8b70-5ccc4e64ad9c admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:06.064425 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 785/1572] 10.222.0.102 () {58 vars in 1158 bytes} [Thu Sep 1 17:31:06 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:31:06.065159 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9a449c-e62a-4699-9b3c-75466e42af74 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:06.069812 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace6e55f-ed66-4591-927b-94404aeb6ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:06.070422 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace6e55f-ed66-4591-927b-94404aeb6ecc admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:06.070706 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace6e55f-ed66-4591-927b-94404aeb6ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:06.070977 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace6e55f-ed66-4591-927b-94404aeb6ecc admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:06.071782 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace6e55f-ed66-4591-927b-94404aeb6ecc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:06.073910 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace6e55f-ed66-4591-927b-94404aeb6ecc admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:06.075630 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9a449c-e62a-4699-9b3c-75466e42af74 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:06.080289 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3e098d-c91e-4698-afe1-9f7348e72c54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:06.080891 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3e098d-c91e-4698-afe1-9f7348e72c54 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:06.081187 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3e098d-c91e-4698-afe1-9f7348e72c54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:06.081497 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3e098d-c91e-4698-afe1-9f7348e72c54 admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:06.082127 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3e098d-c91e-4698-afe1-9f7348e72c54 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:31:06.082426 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3e098d-c91e-4698-afe1-9f7348e72c54 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:06.084025 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3e098d-c91e-4698-afe1-9f7348e72c54 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:06.087555 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 788/1573] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:31:06 2022] GET /identity/v3/roles => generated 1427 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:31:06.087555 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace6e55f-ed66-4591-927b-94404aeb6ecc admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:06.095755 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace6e55f-ed66-4591-927b-94404aeb6ecc admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:06.099575 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 786/1574] 10.222.0.102 () {58 vars in 1158 bytes} [Thu Sep 1 17:31:06 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:31:06.100376 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3e098d-c91e-4698-afe1-9f7348e72c54 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:06.100627 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6001c9-9e24-49c2-8fa2-793fdfb46f5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:06.101245 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6001c9-9e24-49c2-8fa2-793fdfb46f5a admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:06.101535 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6001c9-9e24-49c2-8fa2-793fdfb46f5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:06.101820 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6001c9-9e24-49c2-8fa2-793fdfb46f5a admin admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:06.102455 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6001c9-9e24-49c2-8fa2-793fdfb46f5a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:31:06.102764 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6001c9-9e24-49c2-8fa2-793fdfb46f5a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:06.104393 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6001c9-9e24-49c2-8fa2-793fdfb46f5a admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:06.112596 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3e098d-c91e-4698-afe1-9f7348e72c54 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:06.117814 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b68f5ab-530f-4a31-a01a-f28ad0fc2e37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:06.118020 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b68f5ab-530f-4a31-a01a-f28ad0fc2e37 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:06.118299 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b68f5ab-530f-4a31-a01a-f28ad0fc2e37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:06.118574 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b68f5ab-530f-4a31-a01a-f28ad0fc2e37 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:06.118700 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 789/1575] 10.222.0.102 () {58 vars in 1158 bytes} [Thu Sep 1 17:31:06 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:31:06.119488 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b68f5ab-530f-4a31-a01a-f28ad0fc2e37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:06.121418 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b68f5ab-530f-4a31-a01a-f28ad0fc2e37 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:06.122276 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6001c9-9e24-49c2-8fa2-793fdfb46f5a admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:06.132184 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6001c9-9e24-49c2-8fa2-793fdfb46f5a admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:06.137775 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec3c3e3-6383-413f-be92-6099d5b3a45c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:06.137967 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec3c3e3-6383-413f-be92-6099d5b3a45c admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:06.138246 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec3c3e3-6383-413f-be92-6099d5b3a45c admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:06.138562 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec3c3e3-6383-413f-be92-6099d5b3a45c admin admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:06.139208 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec3c3e3-6383-413f-be92-6099d5b3a45c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:31:06.139516 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec3c3e3-6383-413f-be92-6099d5b3a45c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:06.141688 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec3c3e3-6383-413f-be92-6099d5b3a45c admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:06.159863 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 787/1576] 10.222.0.102 () {60 vars in 1158 bytes} [Thu Sep 1 17:31:06 2022] POST /identity/v3/projects => generated 364 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:31:06.160127 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 790/1577] 10.222.0.102 () {58 vars in 1158 bytes} [Thu Sep 1 17:31:06 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:31:06.160127 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec3c3e3-6383-413f-be92-6099d5b3a45c admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:06.160496 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b68f5ab-530f-4a31-a01a-f28ad0fc2e37 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:06.170854 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec3c3e3-6383-413f-be92-6099d5b3a45c admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:06.225187 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b68f5ab-530f-4a31-a01a-f28ad0fc2e37 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:06.229594 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94191b87-e9de-4ccc-9a44-e5aceca01580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:06.229966 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a42cec-7fe6-4a19-a8e7-65f29f602ea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:06.230202 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94191b87-e9de-4ccc-9a44-e5aceca01580 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:06.230477 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94191b87-e9de-4ccc-9a44-e5aceca01580 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:06.230683 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a42cec-7fe6-4a19-a8e7-65f29f602ea4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:06.230798 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94191b87-e9de-4ccc-9a44-e5aceca01580 admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:06.230986 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a42cec-7fe6-4a19-a8e7-65f29f602ea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:06.231249 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a42cec-7fe6-4a19-a8e7-65f29f602ea4 admin admin] PATH_INFO: `/v3/projects/ad7f8bbb145f4289a09ed9a1bc5a8dfd/users/217cc16ec388405cab13ef82277fc0cb/roles/b75cb360925e4ba5a65f14965c103ffd` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:06.231350 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94191b87-e9de-4ccc-9a44-e5aceca01580 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:06.233264 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94191b87-e9de-4ccc-9a44-e5aceca01580 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:06.246038 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a42cec-7fe6-4a19-a8e7-65f29f602ea4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad7f8bbb145f4289a09ed9a1bc5a8dfd, user_id=217cc16ec388405cab13ef82277fc0cb, role_id=b75cb360925e4ba5a65f14965c103ffd)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:06.247074 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: WARNING py.warnings [None req-38a42cec-7fe6-4a19-a8e7-65f29f602ea4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 17:31:06.247074 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: warnings.warn(msg) Sep 01 17:31:06.247074 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: Sep 01 17:31:06.249043 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a42cec-7fe6-4a19-a8e7-65f29f602ea4 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:06.261106 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 791/1578] 10.222.0.102 () {60 vars in 1158 bytes} [Thu Sep 1 17:31:06 2022] POST /identity/v3/projects => generated 366 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:31:06.264628 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 788/1579] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:31:06 2022] PUT /identity/v3/projects/ad7f8bbb145f4289a09ed9a1bc5a8dfd/users/217cc16ec388405cab13ef82277fc0cb/roles/b75cb360925e4ba5a65f14965c103ffd => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:06.264628 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a42cec-7fe6-4a19-a8e7-65f29f602ea4 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:06.266546 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94191b87-e9de-4ccc-9a44-e5aceca01580 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:06.326316 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a42cec-7fe6-4a19-a8e7-65f29f602ea4 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:06.328863 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94191b87-e9de-4ccc-9a44-e5aceca01580 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:06.331484 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f94e805-f076-4210-85a7-c1c059519013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:06.332094 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f94e805-f076-4210-85a7-c1c059519013 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:06.332377 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f94e805-f076-4210-85a7-c1c059519013 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:06.332681 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f94e805-f076-4210-85a7-c1c059519013 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:06.333395 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f94e805-f076-4210-85a7-c1c059519013 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:06.333762 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428f38f1-4d5c-4ec4-82a2-4711220abf01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:06.334364 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428f38f1-4d5c-4ec4-82a2-4711220abf01 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:06.334645 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428f38f1-4d5c-4ec4-82a2-4711220abf01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:06.334921 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428f38f1-4d5c-4ec4-82a2-4711220abf01 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:06.335529 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f94e805-f076-4210-85a7-c1c059519013 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:06.335653 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428f38f1-4d5c-4ec4-82a2-4711220abf01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:06.337120 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428f38f1-4d5c-4ec4-82a2-4711220abf01 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:06.348239 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 792/1580] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:31:06 2022] GET /identity/v3/roles => generated 1427 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:31:06.349188 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428f38f1-4d5c-4ec4-82a2-4711220abf01 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:06.352009 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:06.360321 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428f38f1-4d5c-4ec4-82a2-4711220abf01 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:06.365543 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9207512c-ffa3-4fb3-bbd3-d8a68c7000f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:06.366053 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 789/1581] 10.222.0.102 () {60 vars in 1158 bytes} [Thu Sep 1 17:31:06 2022] POST /identity/v3/projects => generated 374 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:31:06.366236 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9207512c-ffa3-4fb3-bbd3-d8a68c7000f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:06.366710 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f94e805-f076-4210-85a7-c1c059519013 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:06.366906 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9207512c-ffa3-4fb3-bbd3-d8a68c7000f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:06.367248 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9207512c-ffa3-4fb3-bbd3-d8a68c7000f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:06.368395 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9207512c-ffa3-4fb3-bbd3-d8a68c7000f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:06.372094 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9207512c-ffa3-4fb3-bbd3-d8a68c7000f6 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:06.401869 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 793/1582] 10.222.0.102 () {60 vars in 1158 bytes} [Thu Sep 1 17:31:06 2022] POST /identity/v3/projects => generated 380 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:31:06.402696 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9207512c-ffa3-4fb3-bbd3-d8a68c7000f6 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:06.416836 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:06.433245 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f94e805-f076-4210-85a7-c1c059519013 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:06.470333 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9207512c-ffa3-4fb3-bbd3-d8a68c7000f6 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:06.475251 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e017656-51ac-45b8-803e-dc02a70bb21f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:06.475884 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e017656-51ac-45b8-803e-dc02a70bb21f admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:06.476167 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e017656-51ac-45b8-803e-dc02a70bb21f admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:06.476468 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e017656-51ac-45b8-803e-dc02a70bb21f admin admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:06.477141 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e017656-51ac-45b8-803e-dc02a70bb21f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:06.479132 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e017656-51ac-45b8-803e-dc02a70bb21f admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:06.493580 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83725ea1-c31a-47af-ab1e-bd2fdbe20bb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:06.494479 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83725ea1-c31a-47af-ab1e-bd2fdbe20bb7 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:06.494923 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83725ea1-c31a-47af-ab1e-bd2fdbe20bb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:06.495583 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83725ea1-c31a-47af-ab1e-bd2fdbe20bb7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:06.496288 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83725ea1-c31a-47af-ab1e-bd2fdbe20bb7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:06.498270 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83725ea1-c31a-47af-ab1e-bd2fdbe20bb7 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:06.504563 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 794/1583] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:31:06 2022] POST /identity/v3/users => generated 283 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:31:06.504563 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e017656-51ac-45b8-803e-dc02a70bb21f admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:06.512697 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e017656-51ac-45b8-803e-dc02a70bb21f admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:06.520149 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07b0c3b-7558-43bd-ab07-53667451b79c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:06.520757 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07b0c3b-7558-43bd-ab07-53667451b79c admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:06.521037 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07b0c3b-7558-43bd-ab07-53667451b79c admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:06.522086 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07b0c3b-7558-43bd-ab07-53667451b79c admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:06.522986 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07b0c3b-7558-43bd-ab07-53667451b79c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:06.525279 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07b0c3b-7558-43bd-ab07-53667451b79c admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:06.528888 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 790/1584] 10.222.0.102 () {60 vars in 1158 bytes} [Thu Sep 1 17:31:06 2022] POST /identity/v3/projects => generated 390 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:31:06.528888 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83725ea1-c31a-47af-ab1e-bd2fdbe20bb7 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:06.554351 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 795/1585] 10.222.0.102 () {60 vars in 1158 bytes} [Thu Sep 1 17:31:06 2022] POST /identity/v3/projects => generated 384 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:31:06.555644 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa389ed-69ce-4d25-a6c9-08e66b648327 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:06.555933 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa389ed-69ce-4d25-a6c9-08e66b648327 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:06.556204 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa389ed-69ce-4d25-a6c9-08e66b648327 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:06.583950 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-eaa389ed-69ce-4d25-a6c9-08e66b648327 None None] MFA Rules not processed for user `217cc16ec388405cab13ef82277fc0cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:06.585065 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-eaa389ed-69ce-4d25-a6c9-08e66b648327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:06.596798 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83725ea1-c31a-47af-ab1e-bd2fdbe20bb7 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:06.626879 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:06.654613 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 796/1586] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:06 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:06.654613 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa389ed-69ce-4d25-a6c9-08e66b648327 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:06.655793 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096b0375-b759-40ae-a2c1-a71a75a4d6c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:06.656411 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096b0375-b759-40ae-a2c1-a71a75a4d6c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:06.656689 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096b0375-b759-40ae-a2c1-a71a75a4d6c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:06.656957 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096b0375-b759-40ae-a2c1-a71a75a4d6c1 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:06.657637 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096b0375-b759-40ae-a2c1-a71a75a4d6c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:06.659546 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096b0375-b759-40ae-a2c1-a71a75a4d6c1 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:06.663398 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa389ed-69ce-4d25-a6c9-08e66b648327 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:06.668040 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8284b626-0cb6-4b38-9a7e-c75db598adfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:06.668692 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8284b626-0cb6-4b38-9a7e-c75db598adfa admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:06.668974 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8284b626-0cb6-4b38-9a7e-c75db598adfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:06.669249 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8284b626-0cb6-4b38-9a7e-c75db598adfa admin admin] PATH_INFO: `/v3/projects/79c90aaff0a44b278807167fdbd38e48/users/d364c74791fc4cd4af6773433a7a13cc/roles/b75cb360925e4ba5a65f14965c103ffd` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:06.681181 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 791/1587] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:31:06 2022] POST /identity/v3/users => generated 284 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:31:06.682223 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096b0375-b759-40ae-a2c1-a71a75a4d6c1 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:06.686374 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8284b626-0cb6-4b38-9a7e-c75db598adfa admin admin] RBAC: Authorizing `identity:create_grant(project_id=79c90aaff0a44b278807167fdbd38e48, user_id=d364c74791fc4cd4af6773433a7a13cc, role_id=b75cb360925e4ba5a65f14965c103ffd)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:06.689615 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8284b626-0cb6-4b38-9a7e-c75db598adfa admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:06.694996 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096b0375-b759-40ae-a2c1-a71a75a4d6c1 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:06.700257 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c0cecb-84b8-471e-9746-5362213bd323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:06.700877 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c0cecb-84b8-471e-9746-5362213bd323 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:06.701145 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c0cecb-84b8-471e-9746-5362213bd323 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:06.701471 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c0cecb-84b8-471e-9746-5362213bd323 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:06.702099 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c0cecb-84b8-471e-9746-5362213bd323 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:06.703323 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 797/1588] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:31:06 2022] PUT /identity/v3/projects/79c90aaff0a44b278807167fdbd38e48/users/d364c74791fc4cd4af6773433a7a13cc/roles/b75cb360925e4ba5a65f14965c103ffd => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:06.703922 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8284b626-0cb6-4b38-9a7e-c75db598adfa admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:06.704887 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c0cecb-84b8-471e-9746-5362213bd323 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:06.728622 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 792/1589] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:31:06 2022] POST /identity/v3/users => generated 288 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:31:06.728622 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c0cecb-84b8-471e-9746-5362213bd323 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:06.736211 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c0cecb-84b8-471e-9746-5362213bd323 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:06.748823 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:06.768448 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8284b626-0cb6-4b38-9a7e-c75db598adfa admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:06.773676 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2811c0e5-7dde-4778-a1c8-61e0c742ac1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:06.774285 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2811c0e5-7dde-4778-a1c8-61e0c742ac1a admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:06.774589 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2811c0e5-7dde-4778-a1c8-61e0c742ac1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:06.774864 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2811c0e5-7dde-4778-a1c8-61e0c742ac1a admin admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:06.775494 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2811c0e5-7dde-4778-a1c8-61e0c742ac1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:06.777443 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2811c0e5-7dde-4778-a1c8-61e0c742ac1a admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:06.796622 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fd0941-649d-43f0-8b31-498f0c570106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:06.797270 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fd0941-649d-43f0-8b31-498f0c570106 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:06.797725 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fd0941-649d-43f0-8b31-498f0c570106 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:06.798005 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fd0941-649d-43f0-8b31-498f0c570106 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:06.798656 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fd0941-649d-43f0-8b31-498f0c570106 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:06.800290 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fd0941-649d-43f0-8b31-498f0c570106 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:06.801220 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 798/1590] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:31:06 2022] POST /identity/v3/users => generated 291 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:31:06.802263 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2811c0e5-7dde-4778-a1c8-61e0c742ac1a admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:06.811042 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 793/1591] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:31:06 2022] GET /identity/v3/roles => generated 1427 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:31:06.812612 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fd0941-649d-43f0-8b31-498f0c570106 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:06.813328 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2811c0e5-7dde-4778-a1c8-61e0c742ac1a admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:06.818708 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063356ec-8901-451c-a847-f33dda5b6630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:06.819313 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063356ec-8901-451c-a847-f33dda5b6630 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:06.819594 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063356ec-8901-451c-a847-f33dda5b6630 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:06.819864 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063356ec-8901-451c-a847-f33dda5b6630 admin admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:06.820491 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063356ec-8901-451c-a847-f33dda5b6630 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:06.822518 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063356ec-8901-451c-a847-f33dda5b6630 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:06.824541 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fd0941-649d-43f0-8b31-498f0c570106 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:06.829748 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edf5b3d-0baa-40ef-bfd3-d70b8563760c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:06.830379 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edf5b3d-0baa-40ef-bfd3-d70b8563760c admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:06.830661 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edf5b3d-0baa-40ef-bfd3-d70b8563760c admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:06.831162 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edf5b3d-0baa-40ef-bfd3-d70b8563760c admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:06.831796 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edf5b3d-0baa-40ef-bfd3-d70b8563760c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:06.834276 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edf5b3d-0baa-40ef-bfd3-d70b8563760c admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:06.845346 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 799/1592] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:31:06 2022] POST /identity/v3/users => generated 296 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:31:06.845527 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063356ec-8901-451c-a847-f33dda5b6630 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:06.857384 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063356ec-8901-451c-a847-f33dda5b6630 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:06.862240 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 794/1593] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:31:06 2022] POST /identity/v3/users => generated 293 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:31:06.862585 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8f0500-1131-4d7b-a9ec-323c5a819028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:06.863076 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edf5b3d-0baa-40ef-bfd3-d70b8563760c admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:06.863270 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8f0500-1131-4d7b-a9ec-323c5a819028 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:06.863565 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8f0500-1131-4d7b-a9ec-323c5a819028 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:06.863864 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8f0500-1131-4d7b-a9ec-323c5a819028 admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:06.864497 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8f0500-1131-4d7b-a9ec-323c5a819028 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:06.866505 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8f0500-1131-4d7b-a9ec-323c5a819028 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:06.874214 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edf5b3d-0baa-40ef-bfd3-d70b8563760c admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:06.879334 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ce3010-42ad-4014-9bb2-3caf4dc5e229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:06.879931 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ce3010-42ad-4014-9bb2-3caf4dc5e229 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:06.880218 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ce3010-42ad-4014-9bb2-3caf4dc5e229 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:06.880497 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ce3010-42ad-4014-9bb2-3caf4dc5e229 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:06.881185 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ce3010-42ad-4014-9bb2-3caf4dc5e229 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:06.882849 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ce3010-42ad-4014-9bb2-3caf4dc5e229 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:06.894880 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 795/1594] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:31:06 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:31:06.898720 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 800/1595] 10.222.0.102 () {60 vars in 1158 bytes} [Thu Sep 1 17:31:06 2022] POST /identity/v3/projects => generated 388 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:31:06.898870 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e682c4bd-93c2-45b5-ac07-186b9e78a400 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:06.898870 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e682c4bd-93c2-45b5-ac07-186b9e78a400 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:06.899069 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8f0500-1131-4d7b-a9ec-323c5a819028 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:06.899193 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e682c4bd-93c2-45b5-ac07-186b9e78a400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:06.915693 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-e682c4bd-93c2-45b5-ac07-186b9e78a400 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:06.916895 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-e682c4bd-93c2-45b5-ac07-186b9e78a400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:06.960407 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8f0500-1131-4d7b-a9ec-323c5a819028 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:06.965229 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51368de-de67-4f04-9adf-e4b25a428165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:06.965856 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51368de-de67-4f04-9adf-e4b25a428165 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:06.966130 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51368de-de67-4f04-9adf-e4b25a428165 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:06.966491 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51368de-de67-4f04-9adf-e4b25a428165 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:06.967026 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51368de-de67-4f04-9adf-e4b25a428165 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:06.968671 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51368de-de67-4f04-9adf-e4b25a428165 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:06.979331 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 801/1596] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:31:06 2022] GET /identity/v3/roles => generated 1427 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:31:06.979518 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51368de-de67-4f04-9adf-e4b25a428165 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:06.980616 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 796/1597] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:06 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:06.982701 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e682c4bd-93c2-45b5-ac07-186b9e78a400 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:06.994680 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51368de-de67-4f04-9adf-e4b25a428165 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:06.994855 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e682c4bd-93c2-45b5-ac07-186b9e78a400 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:06.999968 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba02d52f-627a-47bb-94a6-5736923ca9e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:07.000116 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df15695-4c4f-427b-97cc-5bae9fbe786b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:07.000614 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba02d52f-627a-47bb-94a6-5736923ca9e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:07.000811 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df15695-4c4f-427b-97cc-5bae9fbe786b admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:07.000961 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba02d52f-627a-47bb-94a6-5736923ca9e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:07.001071 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df15695-4c4f-427b-97cc-5bae9fbe786b admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:07.001169 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba02d52f-627a-47bb-94a6-5736923ca9e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:07.001272 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df15695-4c4f-427b-97cc-5bae9fbe786b admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:07.001802 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba02d52f-627a-47bb-94a6-5736923ca9e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:07.001915 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df15695-4c4f-427b-97cc-5bae9fbe786b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:07.003412 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba02d52f-627a-47bb-94a6-5736923ca9e9 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:07.003598 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df15695-4c4f-427b-97cc-5bae9fbe786b admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:07.015136 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 797/1598] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:31:06 2022] GET /identity/v3/roles => generated 1427 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:31:07.016238 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df15695-4c4f-427b-97cc-5bae9fbe786b admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:07.017338 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 802/1599] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:31:06 2022] GET /identity/v3/roles => generated 1427 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:31:07.018120 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba02d52f-627a-47bb-94a6-5736923ca9e9 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:07.021527 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:07.028396 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df15695-4c4f-427b-97cc-5bae9fbe786b admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:07.028580 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba02d52f-627a-47bb-94a6-5736923ca9e9 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:07.033775 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019eee3d-1688-4413-bf66-9ad071e6e67c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:07.034373 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019eee3d-1688-4413-bf66-9ad071e6e67c admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:07.034501 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6099209-aa30-4cac-9c9c-e52effec5ac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:07.034655 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019eee3d-1688-4413-bf66-9ad071e6e67c admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:07.034927 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019eee3d-1688-4413-bf66-9ad071e6e67c admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:07.035030 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6099209-aa30-4cac-9c9c-e52effec5ac6 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:07.035300 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6099209-aa30-4cac-9c9c-e52effec5ac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:07.035563 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019eee3d-1688-4413-bf66-9ad071e6e67c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:07.035666 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6099209-aa30-4cac-9c9c-e52effec5ac6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:07.036200 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6099209-aa30-4cac-9c9c-e52effec5ac6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:07.037195 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019eee3d-1688-4413-bf66-9ad071e6e67c admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:07.037843 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6099209-aa30-4cac-9c9c-e52effec5ac6 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:07.049225 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 798/1600] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:31:07 2022] GET /identity/v3/roles => generated 1427 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:31:07.052544 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 803/1601] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:31:07 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:31:07.052544 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019eee3d-1688-4413-bf66-9ad071e6e67c admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:07.053721 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6099209-aa30-4cac-9c9c-e52effec5ac6 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:07.061243 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019eee3d-1688-4413-bf66-9ad071e6e67c admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:07.064586 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6099209-aa30-4cac-9c9c-e52effec5ac6 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:07.066540 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58900d06-7b6a-492c-988e-b7d876aea9ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:07.067135 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58900d06-7b6a-492c-988e-b7d876aea9ad admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:07.067418 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58900d06-7b6a-492c-988e-b7d876aea9ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:07.067711 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58900d06-7b6a-492c-988e-b7d876aea9ad admin admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:07.068319 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58900d06-7b6a-492c-988e-b7d876aea9ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:07.069075 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb72eb4-e4bf-4749-9bcb-80a319c8c20e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:07.069725 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb72eb4-e4bf-4749-9bcb-80a319c8c20e admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:07.070005 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb72eb4-e4bf-4749-9bcb-80a319c8c20e admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:07.070273 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58900d06-7b6a-492c-988e-b7d876aea9ad admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:07.070449 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb72eb4-e4bf-4749-9bcb-80a319c8c20e admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:07.071309 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb72eb4-e4bf-4749-9bcb-80a319c8c20e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:07.073447 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb72eb4-e4bf-4749-9bcb-80a319c8c20e admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:07.086785 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 799/1602] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:31:07 2022] GET /identity/v3/roles => generated 1427 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:31:07.088329 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb72eb4-e4bf-4749-9bcb-80a319c8c20e admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:07.098782 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 804/1603] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:31:07 2022] POST /identity/v3/users => generated 301 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:31:07.099493 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58900d06-7b6a-492c-988e-b7d876aea9ad admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:07.100210 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb72eb4-e4bf-4749-9bcb-80a319c8c20e admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:07.105424 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0989ad-6956-42d7-8586-01fc8fcc3a34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:07.106196 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0989ad-6956-42d7-8586-01fc8fcc3a34 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:07.106486 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0989ad-6956-42d7-8586-01fc8fcc3a34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:07.106762 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0989ad-6956-42d7-8586-01fc8fcc3a34 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:07.107405 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0989ad-6956-42d7-8586-01fc8fcc3a34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:07.109727 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0989ad-6956-42d7-8586-01fc8fcc3a34 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:07.123940 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 800/1604] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:31:07 2022] GET /identity/v3/roles => generated 1427 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:31:07.126818 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0989ad-6956-42d7-8586-01fc8fcc3a34 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:07.137979 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0989ad-6956-42d7-8586-01fc8fcc3a34 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:07.142942 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c0b525-3584-454e-b069-15994327019a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:07.143645 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c0b525-3584-454e-b069-15994327019a admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:07.143925 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c0b525-3584-454e-b069-15994327019a admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:07.144407 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c0b525-3584-454e-b069-15994327019a admin admin] PATH_INFO: `/v3/projects/021f79f41d384ac9ba1f6b0756aa580a/users/59f88fd43f564c4abfe06c0faca536cb/roles/b75cb360925e4ba5a65f14965c103ffd` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:07.167316 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c0b525-3584-454e-b069-15994327019a admin admin] RBAC: Authorizing `identity:create_grant(project_id=021f79f41d384ac9ba1f6b0756aa580a, user_id=59f88fd43f564c4abfe06c0faca536cb, role_id=b75cb360925e4ba5a65f14965c103ffd)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:07.172304 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c0b525-3584-454e-b069-15994327019a admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:07.175443 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:07.180352 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58900d06-7b6a-492c-988e-b7d876aea9ad admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:07.185135 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e51696-6ad9-4dc1-9800-0abcb1357f54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:07.186025 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e51696-6ad9-4dc1-9800-0abcb1357f54 service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:07.186638 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e51696-6ad9-4dc1-9800-0abcb1357f54 service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:07.187026 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e51696-6ad9-4dc1-9800-0abcb1357f54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:07.193160 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 801/1605] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:31:07 2022] PUT /identity/v3/projects/021f79f41d384ac9ba1f6b0756aa580a/users/59f88fd43f564c4abfe06c0faca536cb/roles/b75cb360925e4ba5a65f14965c103ffd => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:07.202480 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c0b525-3584-454e-b069-15994327019a admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:07.206327 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e51696-6ad9-4dc1-9800-0abcb1357f54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:07.207810 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e51696-6ad9-4dc1-9800-0abcb1357f54 service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:07.217914 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 805/1606] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:31:07 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:31:07.218814 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e51696-6ad9-4dc1-9800-0abcb1357f54 service neutron] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:07.230069 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e51696-6ad9-4dc1-9800-0abcb1357f54 service neutron] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:07.278485 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c0b525-3584-454e-b069-15994327019a admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:07.283451 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5de3d1-f9b4-42e5-9a2e-329657cd146f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:07.284115 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5de3d1-f9b4-42e5-9a2e-329657cd146f admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:07.284399 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5de3d1-f9b4-42e5-9a2e-329657cd146f admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:07.284675 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5de3d1-f9b4-42e5-9a2e-329657cd146f admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:07.285355 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5de3d1-f9b4-42e5-9a2e-329657cd146f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:07.287109 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5de3d1-f9b4-42e5-9a2e-329657cd146f admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:07.300758 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 802/1607] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:31:07 2022] GET /identity/v3/roles => generated 1427 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:31:07.301864 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5de3d1-f9b4-42e5-9a2e-329657cd146f admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:07.305152 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3b2a0b-6074-4837-898a-abecef650cf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:07.305825 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3b2a0b-6074-4837-898a-abecef650cf5 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:07.306375 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3b2a0b-6074-4837-898a-abecef650cf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:07.306657 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3b2a0b-6074-4837-898a-abecef650cf5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:07.307311 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3b2a0b-6074-4837-898a-abecef650cf5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:07.309116 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3b2a0b-6074-4837-898a-abecef650cf5 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:07.313664 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5de3d1-f9b4-42e5-9a2e-329657cd146f admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:07.319499 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d836d6d-8fac-4a49-94a5-a30a6cf70337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:07.320133 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d836d6d-8fac-4a49-94a5-a30a6cf70337 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:07.320421 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d836d6d-8fac-4a49-94a5-a30a6cf70337 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:07.320704 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d836d6d-8fac-4a49-94a5-a30a6cf70337 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:07.321436 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d836d6d-8fac-4a49-94a5-a30a6cf70337 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:07.321828 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 806/1608] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:31:07 2022] GET /identity/v3/roles => generated 1427 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:31:07.323217 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3b2a0b-6074-4837-898a-abecef650cf5 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:07.323341 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d836d6d-8fac-4a49-94a5-a30a6cf70337 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:07.337261 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 803/1609] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:31:07 2022] GET /identity/v3/roles => generated 1427 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:31:07.338330 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d836d6d-8fac-4a49-94a5-a30a6cf70337 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:07.338643 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3b2a0b-6074-4837-898a-abecef650cf5 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:07.347458 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2265ad87-e0a6-412d-a5ef-f9214f892ccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:07.348384 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2265ad87-e0a6-412d-a5ef-f9214f892ccf admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:07.348974 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2265ad87-e0a6-412d-a5ef-f9214f892ccf admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:07.349339 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2265ad87-e0a6-412d-a5ef-f9214f892ccf admin admin] PATH_INFO: `/v3/projects/94689412ae164432a18b7444b74c2bae/users/8518291fd07847aca6e86828a43deb01/roles/b75cb360925e4ba5a65f14965c103ffd` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:07.351627 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d836d6d-8fac-4a49-94a5-a30a6cf70337 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:07.356998 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c914f3e-cdbd-4fbc-899d-0e573f847f5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:07.357270 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c914f3e-cdbd-4fbc-899d-0e573f847f5a admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:07.357509 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c914f3e-cdbd-4fbc-899d-0e573f847f5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:07.357776 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c914f3e-cdbd-4fbc-899d-0e573f847f5a admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:07.358576 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c914f3e-cdbd-4fbc-899d-0e573f847f5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:07.360973 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c914f3e-cdbd-4fbc-899d-0e573f847f5a admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:07.367239 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2265ad87-e0a6-412d-a5ef-f9214f892ccf admin admin] RBAC: Authorizing `identity:create_grant(project_id=94689412ae164432a18b7444b74c2bae, user_id=8518291fd07847aca6e86828a43deb01, role_id=b75cb360925e4ba5a65f14965c103ffd)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:07.370756 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2265ad87-e0a6-412d-a5ef-f9214f892ccf admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:07.377868 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 804/1610] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:31:07 2022] GET /identity/v3/roles => generated 1427 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:31:07.377868 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c914f3e-cdbd-4fbc-899d-0e573f847f5a admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:07.387418 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c914f3e-cdbd-4fbc-899d-0e573f847f5a admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:07.390215 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 807/1611] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:31:07 2022] PUT /identity/v3/projects/94689412ae164432a18b7444b74c2bae/users/8518291fd07847aca6e86828a43deb01/roles/b75cb360925e4ba5a65f14965c103ffd => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:07.391618 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dcd1ad-0133-4e0b-a113-1804b39fe7a5 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:07.391904 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dcd1ad-0133-4e0b-a113-1804b39fe7a5 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:07.392186 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dcd1ad-0133-4e0b-a113-1804b39fe7a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:07.428361 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-50dcd1ad-0133-4e0b-a113-1804b39fe7a5 None None] MFA Rules not processed for user `59f88fd43f564c4abfe06c0faca536cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:07.430215 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-50dcd1ad-0133-4e0b-a113-1804b39fe7a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:07.471117 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69171a6-26c8-4db1-828b-7530082e48e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:07.472860 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69171a6-26c8-4db1-828b-7530082e48e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:07.473168 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69171a6-26c8-4db1-828b-7530082e48e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:07.473543 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69171a6-26c8-4db1-828b-7530082e48e0 admin admin] PATH_INFO: `/v3/projects/7864c80b2a9b40aeb18e046fe866cf2b/users/628da841395b4b7e9b3d238c04fe7cee/roles/b75cb360925e4ba5a65f14965c103ffd` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:07.507370 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69171a6-26c8-4db1-828b-7530082e48e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7864c80b2a9b40aeb18e046fe866cf2b, user_id=628da841395b4b7e9b3d238c04fe7cee, role_id=b75cb360925e4ba5a65f14965c103ffd)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:07.511103 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69171a6-26c8-4db1-828b-7530082e48e0 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:07.533119 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 805/1612] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:31:07 2022] PUT /identity/v3/projects/7864c80b2a9b40aeb18e046fe866cf2b/users/628da841395b4b7e9b3d238c04fe7cee/roles/b75cb360925e4ba5a65f14965c103ffd => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:07.543420 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 808/1613] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:07.544293 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dcd1ad-0133-4e0b-a113-1804b39fe7a5 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:07.551079 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69171a6-26c8-4db1-828b-7530082e48e0 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:07.556931 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:07.561806 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dcd1ad-0133-4e0b-a113-1804b39fe7a5 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:07.630883 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:07.646871 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fcd0f8-8411-4aff-9c72-80e6048c27bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:07.647571 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fcd0f8-8411-4aff-9c72-80e6048c27bd admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:07.648082 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fcd0f8-8411-4aff-9c72-80e6048c27bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:07.648361 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fcd0f8-8411-4aff-9c72-80e6048c27bd admin admin] PATH_INFO: `/v3/projects/97796d8be4794e89a89965af17bb3fa9/users/26bf17f1fb8948008fed82f4ba33bf66/roles/b75cb360925e4ba5a65f14965c103ffd` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:07.650953 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69171a6-26c8-4db1-828b-7530082e48e0 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:07.659509 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba94c9fe-dae4-4f79-9214-3b621b530318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:07.660220 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba94c9fe-dae4-4f79-9214-3b621b530318 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:07.660533 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba94c9fe-dae4-4f79-9214-3b621b530318 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:07.660811 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba94c9fe-dae4-4f79-9214-3b621b530318 admin admin] PATH_INFO: `/v3/projects/1f232c720f5a4108a46e79700be67099/users/19a53d1e1a84408ea1dc2aba24bd8299/roles/b75cb360925e4ba5a65f14965c103ffd` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:07.670286 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fcd0f8-8411-4aff-9c72-80e6048c27bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=97796d8be4794e89a89965af17bb3fa9, user_id=26bf17f1fb8948008fed82f4ba33bf66, role_id=b75cb360925e4ba5a65f14965c103ffd)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:07.674350 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fcd0f8-8411-4aff-9c72-80e6048c27bd admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:07.679605 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba94c9fe-dae4-4f79-9214-3b621b530318 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f232c720f5a4108a46e79700be67099, user_id=19a53d1e1a84408ea1dc2aba24bd8299, role_id=b75cb360925e4ba5a65f14965c103ffd)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:07.682428 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba94c9fe-dae4-4f79-9214-3b621b530318 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:07.696112 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 809/1614] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:31:07 2022] PUT /identity/v3/projects/97796d8be4794e89a89965af17bb3fa9/users/26bf17f1fb8948008fed82f4ba33bf66/roles/b75cb360925e4ba5a65f14965c103ffd => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:07.697546 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fcd0f8-8411-4aff-9c72-80e6048c27bd admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:07.698541 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 806/1615] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:31:07 2022] PUT /identity/v3/projects/1f232c720f5a4108a46e79700be67099/users/19a53d1e1a84408ea1dc2aba24bd8299/roles/b75cb360925e4ba5a65f14965c103ffd => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:07.699668 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba94c9fe-dae4-4f79-9214-3b621b530318 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:07.773226 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fcd0f8-8411-4aff-9c72-80e6048c27bd admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:07.775272 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba94c9fe-dae4-4f79-9214-3b621b530318 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:07.778364 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896b96ae-e969-4815-8637-e771a2a8c6c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:07.779948 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896b96ae-e969-4815-8637-e771a2a8c6c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:07.780243 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28d8ad4-f74e-4308-97fe-f618f7ec9c8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:07.780394 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896b96ae-e969-4815-8637-e771a2a8c6c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:07.781801 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896b96ae-e969-4815-8637-e771a2a8c6c5 admin admin] PATH_INFO: `/v3/projects/577f304fbcfb4e01ab66bd4a62d30d8e/users/a54660f778c24611ae52c1628d17fd78/roles/b75cb360925e4ba5a65f14965c103ffd` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:07.781918 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28d8ad4-f74e-4308-97fe-f618f7ec9c8d admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:07.782052 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28d8ad4-f74e-4308-97fe-f618f7ec9c8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:07.782406 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28d8ad4-f74e-4308-97fe-f618f7ec9c8d admin admin] PATH_INFO: `/v3/projects/80bbf3425b2546e48aac5d74cd289cdd/users/2bed612da06848f19a468a8b9aeb3db5/roles/36577a6f46d642d4b81805ede9872490` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:07.802070 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28d8ad4-f74e-4308-97fe-f618f7ec9c8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=80bbf3425b2546e48aac5d74cd289cdd, user_id=2bed612da06848f19a468a8b9aeb3db5, role_id=36577a6f46d642d4b81805ede9872490)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:07.804803 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28d8ad4-f74e-4308-97fe-f618f7ec9c8d admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:07.807156 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896b96ae-e969-4815-8637-e771a2a8c6c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=577f304fbcfb4e01ab66bd4a62d30d8e, user_id=a54660f778c24611ae52c1628d17fd78, role_id=b75cb360925e4ba5a65f14965c103ffd)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:07.811526 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896b96ae-e969-4815-8637-e771a2a8c6c5 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:07.825658 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 807/1616] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:31:07 2022] PUT /identity/v3/projects/80bbf3425b2546e48aac5d74cd289cdd/users/2bed612da06848f19a468a8b9aeb3db5/roles/36577a6f46d642d4b81805ede9872490 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:07.827003 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d369c5a-7bff-4e05-868d-a360fe310160 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:07.827298 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d369c5a-7bff-4e05-868d-a360fe310160 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:07.828098 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d369c5a-7bff-4e05-868d-a360fe310160 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:07.832547 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 810/1617] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:31:07 2022] PUT /identity/v3/projects/577f304fbcfb4e01ab66bd4a62d30d8e/users/a54660f778c24611ae52c1628d17fd78/roles/b75cb360925e4ba5a65f14965c103ffd => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:07.833673 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896b96ae-e969-4815-8637-e771a2a8c6c5 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:07.869498 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-8d369c5a-7bff-4e05-868d-a360fe310160 None None] MFA Rules not processed for user `8518291fd07847aca6e86828a43deb01`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:07.871392 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-8d369c5a-7bff-4e05-868d-a360fe310160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:07.909360 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896b96ae-e969-4815-8637-e771a2a8c6c5 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:07.915259 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260272cc-931e-4830-815a-e4bec32bac75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:07.915984 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260272cc-931e-4830-815a-e4bec32bac75 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:07.916368 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260272cc-931e-4830-815a-e4bec32bac75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:07.916898 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260272cc-931e-4830-815a-e4bec32bac75 admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:07.917770 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260272cc-931e-4830-815a-e4bec32bac75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:07.919832 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260272cc-931e-4830-815a-e4bec32bac75 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:07.946509 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:07.953385 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 811/1618] 10.222.0.102 () {60 vars in 1158 bytes} [Thu Sep 1 17:31:07 2022] POST /identity/v3/projects => generated 366 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:31:07.954201 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998f4efc-634e-4098-87b1-041a99e94b0c None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:07.954448 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998f4efc-634e-4098-87b1-041a99e94b0c None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:07.954727 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998f4efc-634e-4098-87b1-041a99e94b0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:07.964129 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 808/1619] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:07 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:07.964129 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57248d9-c02c-4ddb-b4e5-56b1dd2b5419 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:07.964129 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57248d9-c02c-4ddb-b4e5-56b1dd2b5419 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:07.964129 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57248d9-c02c-4ddb-b4e5-56b1dd2b5419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:07.988497 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-c57248d9-c02c-4ddb-b4e5-56b1dd2b5419 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:07.989771 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-c57248d9-c02c-4ddb-b4e5-56b1dd2b5419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:07.993131 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-998f4efc-634e-4098-87b1-041a99e94b0c None None] MFA Rules not processed for user `628da841395b4b7e9b3d238c04fe7cee`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:07.994677 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-998f4efc-634e-4098-87b1-041a99e94b0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:08.039665 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:08.088946 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 809/1620] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:07 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:08.090546 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 812/1621] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:07 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:08.090979 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563b1b47-5315-4316-92cb-2f14c112e1ed None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:08.091411 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998f4efc-634e-4098-87b1-041a99e94b0c None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:08.098663 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563b1b47-5315-4316-92cb-2f14c112e1ed None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:08.099365 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563b1b47-5315-4316-92cb-2f14c112e1ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:08.103456 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998f4efc-634e-4098-87b1-041a99e94b0c None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:08.108516 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b9a84b-51a8-4c17-b6f7-5e743cd25684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:08.109251 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b9a84b-51a8-4c17-b6f7-5e743cd25684 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:08.109623 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b9a84b-51a8-4c17-b6f7-5e743cd25684 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:08.109902 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b9a84b-51a8-4c17-b6f7-5e743cd25684 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:08.111505 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b9a84b-51a8-4c17-b6f7-5e743cd25684 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:08.113469 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b9a84b-51a8-4c17-b6f7-5e743cd25684 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:08.127176 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-563b1b47-5315-4316-92cb-2f14c112e1ed None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:08.129086 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-563b1b47-5315-4316-92cb-2f14c112e1ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:08.134769 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 813/1622] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:31:08 2022] GET /identity/v3/roles => generated 1427 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:31:08.136600 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2e6910-7e0b-4593-8e60-31e426c41cde None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:08.136895 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2e6910-7e0b-4593-8e60-31e426c41cde None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:08.137172 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2e6910-7e0b-4593-8e60-31e426c41cde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:08.138787 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 810/1623] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:08 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:08.139731 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563b1b47-5315-4316-92cb-2f14c112e1ed None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:08.162530 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563b1b47-5315-4316-92cb-2f14c112e1ed None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:08.170170 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2918bedb-55ed-4a23-ba37-6d571032f0e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:08.173168 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2918bedb-55ed-4a23-ba37-6d571032f0e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:08.173594 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2918bedb-55ed-4a23-ba37-6d571032f0e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:08.173962 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2918bedb-55ed-4a23-ba37-6d571032f0e0 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:08.174702 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2918bedb-55ed-4a23-ba37-6d571032f0e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:08.177355 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2918bedb-55ed-4a23-ba37-6d571032f0e0 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:08.195559 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-ba2e6910-7e0b-4593-8e60-31e426c41cde None None] MFA Rules not processed for user `a54660f778c24611ae52c1628d17fd78`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:08.196944 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-ba2e6910-7e0b-4593-8e60-31e426c41cde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:08.212769 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 811/1624] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:31:08 2022] POST /identity/v3/users => generated 290 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:31:08.214562 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2918bedb-55ed-4a23-ba37-6d571032f0e0 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:08.284771 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2918bedb-55ed-4a23-ba37-6d571032f0e0 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:08.285574 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 814/1625] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:08 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:08.286286 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2e6910-7e0b-4593-8e60-31e426c41cde None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:08.300164 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18677e8e-22fc-4b53-a278-7e5f9a64e9fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '523ed589872a45a09b6a7675a7fb0b75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:08.307456 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18677e8e-22fc-4b53-a278-7e5f9a64e9fc service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:08.307911 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18677e8e-22fc-4b53-a278-7e5f9a64e9fc service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:08.308285 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18677e8e-22fc-4b53-a278-7e5f9a64e9fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:08.368973 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2e6910-7e0b-4593-8e60-31e426c41cde None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:08.376312 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600c375f-b3e0-4dfe-8036-906bfb4828c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:08.376995 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600c375f-b3e0-4dfe-8036-906bfb4828c6 service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:08.377501 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600c375f-b3e0-4dfe-8036-906bfb4828c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:08.377873 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600c375f-b3e0-4dfe-8036-906bfb4828c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:08.393844 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600c375f-b3e0-4dfe-8036-906bfb4828c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:08.396994 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600c375f-b3e0-4dfe-8036-906bfb4828c6 service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:08.400621 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18677e8e-22fc-4b53-a278-7e5f9a64e9fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:08.402558 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18677e8e-22fc-4b53-a278-7e5f9a64e9fc service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:08.410250 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 815/1626] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:31:08 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:31:08.412596 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600c375f-b3e0-4dfe-8036-906bfb4828c6 service neutron] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:08.413500 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 812/1627] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:31:08 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:31:08.425022 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600c375f-b3e0-4dfe-8036-906bfb4828c6 service neutron] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:08.428232 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5535f63d-2d5e-4555-a716-3095a64ea937 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:08.430198 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5535f63d-2d5e-4555-a716-3095a64ea937 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:08.430341 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5535f63d-2d5e-4555-a716-3095a64ea937 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:08.431852 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13a9786-046d-45ba-9617-a0a3952bc587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:08.432908 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13a9786-046d-45ba-9617-a0a3952bc587 admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:08.433366 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13a9786-046d-45ba-9617-a0a3952bc587 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:08.433801 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13a9786-046d-45ba-9617-a0a3952bc587 admin admin] PATH_INFO: `/v3/projects/80bbf3425b2546e48aac5d74cd289cdd/users/2bed612da06848f19a468a8b9aeb3db5/roles/36577a6f46d642d4b81805ede9872490` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:08.436007 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13a9786-046d-45ba-9617-a0a3952bc587 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80bbf3425b2546e48aac5d74cd289cdd, user_id=2bed612da06848f19a468a8b9aeb3db5, role_id=36577a6f46d642d4b81805ede9872490)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:08.439000 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 813/1628] 10.222.0.102 () {54 vars in 967 bytes} [Thu Sep 1 17:31:08 2022] GET /identity => generated 271 bytes in 25 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:31:08.440819 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5535f63d-2d5e-4555-a716-3095a64ea937 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:08.441206 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13a9786-046d-45ba-9617-a0a3952bc587 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:08.459205 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5535f63d-2d5e-4555-a716-3095a64ea937 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:08.461732 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 816/1629] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:31:08 2022] PUT /identity/v3/projects/80bbf3425b2546e48aac5d74cd289cdd/users/2bed612da06848f19a468a8b9aeb3db5/roles/36577a6f46d642d4b81805ede9872490 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:08.463120 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13a9786-046d-45ba-9617-a0a3952bc587 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:08.503222 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:08.531826 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c03befd-8356-4ae4-9591-34e4ea17d169 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:08.532869 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c03befd-8356-4ae4-9591-34e4ea17d169 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:08.533151 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c03befd-8356-4ae4-9591-34e4ea17d169 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:08.533564 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c03befd-8356-4ae4-9591-34e4ea17d169 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:08.578905 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:08.609534 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c03befd-8356-4ae4-9591-34e4ea17d169 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:08.611702 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c03befd-8356-4ae4-9591-34e4ea17d169 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:08.624008 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 814/1630] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:31:08 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:31:08.625500 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13a9786-046d-45ba-9617-a0a3952bc587 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:08.626134 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886ec68a-efdf-4b3c-9fcc-69758a234db2 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:08.626800 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886ec68a-efdf-4b3c-9fcc-69758a234db2 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:08.627199 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886ec68a-efdf-4b3c-9fcc-69758a234db2 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:08.636240 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 815/1631] 10.222.0.102 () {54 vars in 967 bytes} [Thu Sep 1 17:31:08 2022] GET /identity => generated 271 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:31:08.638210 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543385b5-fffe-48fd-87e8-925c46002f8e None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:08.638693 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543385b5-fffe-48fd-87e8-925c46002f8e None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:08.639244 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543385b5-fffe-48fd-87e8-925c46002f8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:08.641938 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c97412-9689-4f42-ad3b-5eb31445acd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:08.642572 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c97412-9689-4f42-ad3b-5eb31445acd9 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:08.642938 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c97412-9689-4f42-ad3b-5eb31445acd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:08.643114 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c97412-9689-4f42-ad3b-5eb31445acd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:08.643761 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c97412-9689-4f42-ad3b-5eb31445acd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:08.645474 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c97412-9689-4f42-ad3b-5eb31445acd9 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:08.677484 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 817/1632] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:31:08 2022] GET /identity/v3/roles => generated 1427 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:31:08.680096 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29804c95-1490-4604-84ee-c2527e968f1d None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:08.680819 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29804c95-1490-4604-84ee-c2527e968f1d None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:08.681098 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29804c95-1490-4604-84ee-c2527e968f1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:08.690354 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-543385b5-fffe-48fd-87e8-925c46002f8e None None] MFA Rules not processed for user `5f8293316baf4774b62fc11d55d50865`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:08.691742 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-543385b5-fffe-48fd-87e8-925c46002f8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:08.749753 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-29804c95-1490-4604-84ee-c2527e968f1d None None] MFA Rules not processed for user `2bed612da06848f19a468a8b9aeb3db5`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:08.751544 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-29804c95-1490-4604-84ee-c2527e968f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:08.833599 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 816/1633] 10.222.0.102 () {58 vars in 1064 bytes} [Thu Sep 1 17:31:08 2022] POST /identity/v3/auth/tokens => generated 3382 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:08.835627 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5f914c-ea02-424d-9f6c-38ae47fb3a63 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:08.835988 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5f914c-ea02-424d-9f6c-38ae47fb3a63 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:08.836421 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5f914c-ea02-424d-9f6c-38ae47fb3a63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:08.871194 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-fb5f914c-ea02-424d-9f6c-38ae47fb3a63 None None] MFA Rules not processed for user `5f8293316baf4774b62fc11d55d50865`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:08.873186 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-fb5f914c-ea02-424d-9f6c-38ae47fb3a63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:08.897335 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 817/1634] 10.222.0.102 () {58 vars in 1064 bytes} [Thu Sep 1 17:31:08 2022] POST /identity/v3/auth/tokens => generated 3382 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:08.898499 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5f914c-ea02-424d-9f6c-38ae47fb3a63 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:08.923289 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5f914c-ea02-424d-9f6c-38ae47fb3a63 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:08.942312 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 818/1635] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:08 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:08.945042 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29804c95-1490-4604-84ee-c2527e968f1d None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:08.947349 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974cdb7b-b37f-440f-884c-7305a0a96476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:08.949438 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974cdb7b-b37f-440f-884c-7305a0a96476 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:08.949842 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974cdb7b-b37f-440f-884c-7305a0a96476 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:08.950232 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974cdb7b-b37f-440f-884c-7305a0a96476 admin admin] PATH_INFO: `/v3/projects/0dca75e858bb4bb18bec335728d01390/users/9e20cf58fdd841128b677e5d7bd0757e/roles/36577a6f46d642d4b81805ede9872490` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:08.974579 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29804c95-1490-4604-84ee-c2527e968f1d None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:09.007961 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9848389-c9fe-4f91-a309-87d95a1769b7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f8293316baf4774b62fc11d55d50865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:09.009772 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9848389-c9fe-4f91-a309-87d95a1769b7 service nova] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:09.010054 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974cdb7b-b37f-440f-884c-7305a0a96476 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0dca75e858bb4bb18bec335728d01390, user_id=9e20cf58fdd841128b677e5d7bd0757e, role_id=36577a6f46d642d4b81805ede9872490)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:09.010195 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9848389-c9fe-4f91-a309-87d95a1769b7 service nova] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:09.012931 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9848389-c9fe-4f91-a309-87d95a1769b7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:09.017782 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974cdb7b-b37f-440f-884c-7305a0a96476 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:09.050908 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 818/1636] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:31:08 2022] PUT /identity/v3/projects/0dca75e858bb4bb18bec335728d01390/users/9e20cf58fdd841128b677e5d7bd0757e/roles/36577a6f46d642d4b81805ede9872490 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:09.052270 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974cdb7b-b37f-440f-884c-7305a0a96476 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:09.118100 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:09.172212 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9848389-c9fe-4f91-a309-87d95a1769b7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:09.173946 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9848389-c9fe-4f91-a309-87d95a1769b7 service nova] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:09.192197 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974cdb7b-b37f-440f-884c-7305a0a96476 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:09.202837 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45f55cf-e684-4b29-878b-6c7cd7f3c46a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f8293316baf4774b62fc11d55d50865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:09.203976 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45f55cf-e684-4b29-878b-6c7cd7f3c46a service nova] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:09.204366 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45f55cf-e684-4b29-878b-6c7cd7f3c46a service nova] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:09.204833 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45f55cf-e684-4b29-878b-6c7cd7f3c46a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:09.206356 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 819/1637] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:31:08 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:31:09.207356 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9848389-c9fe-4f91-a309-87d95a1769b7 service nova] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:09.293038 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45f55cf-e684-4b29-878b-6c7cd7f3c46a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:09.295176 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45f55cf-e684-4b29-878b-6c7cd7f3c46a service nova] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:09.310656 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 819/1638] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:31:09 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:31:09.312963 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45f55cf-e684-4b29-878b-6c7cd7f3c46a service nova] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:09.428682 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9848389-c9fe-4f91-a309-87d95a1769b7 service nova] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:09.526284 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45f55cf-e684-4b29-878b-6c7cd7f3c46a service nova] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:09.526284 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de32581-caf4-4893-a80a-5bab7065e7a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:09.526284 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de32581-caf4-4893-a80a-5bab7065e7a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:09.526284 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de32581-caf4-4893-a80a-5bab7065e7a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:09.527023 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de32581-caf4-4893-a80a-5bab7065e7a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:09.527023 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de32581-caf4-4893-a80a-5bab7065e7a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:09.527023 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de32581-caf4-4893-a80a-5bab7065e7a7 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:09.527023 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 820/1639] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:31:09 2022] GET /identity/v3/roles => generated 1427 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:31:09.527023 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de32581-caf4-4893-a80a-5bab7065e7a7 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:09.527023 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de32581-caf4-4893-a80a-5bab7065e7a7 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:09.527545 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:09.527657 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fc65a0-d7bd-4001-84b9-89fdf1798dfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:09.527657 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fc65a0-d7bd-4001-84b9-89fdf1798dfd admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:09.527657 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fc65a0-d7bd-4001-84b9-89fdf1798dfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:09.527657 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fc65a0-d7bd-4001-84b9-89fdf1798dfd admin admin] PATH_INFO: `/v3/projects/0dca75e858bb4bb18bec335728d01390/users/9e20cf58fdd841128b677e5d7bd0757e/roles/36577a6f46d642d4b81805ede9872490` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:09.527953 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fc65a0-d7bd-4001-84b9-89fdf1798dfd admin admin] RBAC: Authorizing `identity:create_grant(project_id=0dca75e858bb4bb18bec335728d01390, user_id=9e20cf58fdd841128b677e5d7bd0757e, role_id=36577a6f46d642d4b81805ede9872490)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:09.529049 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fc65a0-d7bd-4001-84b9-89fdf1798dfd admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:09.546509 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 821/1640] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:31:09 2022] PUT /identity/v3/projects/0dca75e858bb4bb18bec335728d01390/users/9e20cf58fdd841128b677e5d7bd0757e/roles/36577a6f46d642d4b81805ede9872490 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:09.570017 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60696204-25ed-49f9-8597-edf805f04ba6 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:09.570834 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60696204-25ed-49f9-8597-edf805f04ba6 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:09.571341 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60696204-25ed-49f9-8597-edf805f04ba6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:09.662482 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-60696204-25ed-49f9-8597-edf805f04ba6 None None] MFA Rules not processed for user `9e20cf58fdd841128b677e5d7bd0757e`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:09.664426 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-60696204-25ed-49f9-8597-edf805f04ba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:09.691964 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8339ae9d-8d45-43e7-985e-c9dc19d2b153 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '523ed589872a45a09b6a7675a7fb0b75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:09.693230 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8339ae9d-8d45-43e7-985e-c9dc19d2b153 service cinder] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:09.693497 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8339ae9d-8d45-43e7-985e-c9dc19d2b153 service cinder] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:09.693774 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8339ae9d-8d45-43e7-985e-c9dc19d2b153 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:09.767172 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:09.919449 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 822/1641] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:09 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 356 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:09.920821 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432f9b4b-8eae-4eb1-b49d-5bc040503b67 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:09.921209 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432f9b4b-8eae-4eb1-b49d-5bc040503b67 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:09.926332 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432f9b4b-8eae-4eb1-b49d-5bc040503b67 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:09.930614 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 823/1642] 10.222.0.102 () {54 vars in 917 bytes} [Thu Sep 1 17:31:09 2022] GET /identity => generated 271 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:31:09.934597 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432f9b4b-8eae-4eb1-b49d-5bc040503b67 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:09.962678 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8339ae9d-8d45-43e7-985e-c9dc19d2b153 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:09.966513 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8339ae9d-8d45-43e7-985e-c9dc19d2b153 service cinder] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:09.990189 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 820/1643] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:31:09 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 783 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:31:10.006456 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325105d6-851a-4a55-b42d-71d7747e1cdf None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:10.006752 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325105d6-851a-4a55-b42d-71d7747e1cdf None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:10.007032 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325105d6-851a-4a55-b42d-71d7747e1cdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:10.056044 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-325105d6-851a-4a55-b42d-71d7747e1cdf None None] MFA Rules not processed for user `5f8293316baf4774b62fc11d55d50865`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:10.058968 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-325105d6-851a-4a55-b42d-71d7747e1cdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:10.090656 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432f9b4b-8eae-4eb1-b49d-5bc040503b67 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:10.099791 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977e2e1f-6ba6-445b-9467-fc19e35a7124 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '523ed589872a45a09b6a7675a7fb0b75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:10.101762 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977e2e1f-6ba6-445b-9467-fc19e35a7124 service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:10.102675 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977e2e1f-6ba6-445b-9467-fc19e35a7124 service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:10.102994 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977e2e1f-6ba6-445b-9467-fc19e35a7124 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:10.124722 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977e2e1f-6ba6-445b-9467-fc19e35a7124 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:10.126303 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977e2e1f-6ba6-445b-9467-fc19e35a7124 service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:10.140988 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 824/1644] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:31:09 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:31:10.155059 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5213b592-0094-4622-b067-db28e13df965 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:10.155301 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5213b592-0094-4622-b067-db28e13df965 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:10.164236 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5213b592-0094-4622-b067-db28e13df965 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:10.164236 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 825/1645] 10.222.0.102 () {54 vars in 917 bytes} [Thu Sep 1 17:31:10 2022] GET /identity => generated 271 bytes in 20 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:31:10.164236 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5213b592-0094-4622-b067-db28e13df965 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:10.256905 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 821/1646] 10.222.0.102 () {58 vars in 1014 bytes} [Thu Sep 1 17:31:09 2022] POST /identity/v3/auth/tokens => generated 3382 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:10.259137 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226eb4c3-3a56-4ee7-8911-04003db7a448 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:10.259881 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226eb4c3-3a56-4ee7-8911-04003db7a448 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:10.262236 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226eb4c3-3a56-4ee7-8911-04003db7a448 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:10.344761 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-226eb4c3-3a56-4ee7-8911-04003db7a448 None None] MFA Rules not processed for user `5f8293316baf4774b62fc11d55d50865`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:10.346050 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-226eb4c3-3a56-4ee7-8911-04003db7a448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:10.365927 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 822/1647] 10.222.0.102 () {58 vars in 1014 bytes} [Thu Sep 1 17:31:10 2022] POST /identity/v3/auth/tokens => generated 3382 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:10.366821 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226eb4c3-3a56-4ee7-8911-04003db7a448 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:10.375126 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5213b592-0094-4622-b067-db28e13df965 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:10.387033 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226eb4c3-3a56-4ee7-8911-04003db7a448 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:10.390541 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc6640d-e0eb-41e4-a169-e4d1c7dda7b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:10.391213 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc6640d-e0eb-41e4-a169-e4d1c7dda7b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:10.391493 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc6640d-e0eb-41e4-a169-e4d1c7dda7b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:10.391764 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc6640d-e0eb-41e4-a169-e4d1c7dda7b3 admin admin] PATH_INFO: `/v3/users/217cc16ec388405cab13ef82277fc0cb` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:10.393061 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc6640d-e0eb-41e4-a169-e4d1c7dda7b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=217cc16ec388405cab13ef82277fc0cb)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:10.394701 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: WARNING py.warnings [None req-4dc6640d-e0eb-41e4-a169-e4d1c7dda7b3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 17:31:10.394701 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: warnings.warn(msg) Sep 01 17:31:10.394701 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: Sep 01 17:31:10.395946 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc6640d-e0eb-41e4-a169-e4d1c7dda7b3 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:10.400793 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd9da56-e08f-49f3-9481-06b464ed3461 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '523ed589872a45a09b6a7675a7fb0b75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:10.401463 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd9da56-e08f-49f3-9481-06b464ed3461 service cinder] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:10.401744 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd9da56-e08f-49f3-9481-06b464ed3461 service cinder] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:10.402018 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd9da56-e08f-49f3-9481-06b464ed3461 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:10.428610 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd9da56-e08f-49f3-9481-06b464ed3461 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:10.430098 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd9da56-e08f-49f3-9481-06b464ed3461 service cinder] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:10.459358 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 823/1648] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:31:10 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:31:10.460512 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd9da56-e08f-49f3-9481-06b464ed3461 service cinder] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:10.478148 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd9da56-e08f-49f3-9481-06b464ed3461 service cinder] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:10.484620 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca7e5b3-65bd-4b03-b3d6-276eea30b215 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '523ed589872a45a09b6a7675a7fb0b75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:10.485376 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca7e5b3-65bd-4b03-b3d6-276eea30b215 service cinder] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:10.485615 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca7e5b3-65bd-4b03-b3d6-276eea30b215 service cinder] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:10.485888 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca7e5b3-65bd-4b03-b3d6-276eea30b215 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:10.546719 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-4dc6640d-e0eb-41e4-a169-e4d1c7dda7b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '217cc16ec388405cab13ef82277fc0cb'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:10.580821 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-4dc6640d-e0eb-41e4-a169-e4d1c7dda7b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '217cc16ec388405cab13ef82277fc0cb'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:10.592578 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-4dc6640d-e0eb-41e4-a169-e4d1c7dda7b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '217cc16ec388405cab13ef82277fc0cb'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:10.594684 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-4dc6640d-e0eb-41e4-a169-e4d1c7dda7b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '217cc16ec388405cab13ef82277fc0cb'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:10.596537 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-4dc6640d-e0eb-41e4-a169-e4d1c7dda7b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '217cc16ec388405cab13ef82277fc0cb'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:10.601977 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:10.612345 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:10.634316 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 826/1649] 10.222.0.102 () {58 vars in 1229 bytes} [Thu Sep 1 17:31:10 2022] DELETE /identity/v3/users/217cc16ec388405cab13ef82277fc0cb => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:10.634316 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc6640d-e0eb-41e4-a169-e4d1c7dda7b3 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:10.634316 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-4dc6640d-e0eb-41e4-a169-e4d1c7dda7b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:10.642186 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca7e5b3-65bd-4b03-b3d6-276eea30b215 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:10.646463 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca7e5b3-65bd-4b03-b3d6-276eea30b215 service cinder] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:10.661901 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 824/1650] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:31:10 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:31:10.663556 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc5763a-5f27-45f4-98df-a5c000da33f8 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:10.664752 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc5763a-5f27-45f4-98df-a5c000da33f8 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:10.665244 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc5763a-5f27-45f4-98df-a5c000da33f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:10.692428 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-8cc5763a-5f27-45f4-98df-a5c000da33f8 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:10.694904 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-8cc5763a-5f27-45f4-98df-a5c000da33f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:10.751848 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc6640d-e0eb-41e4-a169-e4d1c7dda7b3 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:10.758640 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581b6d42-9ddf-49ab-bffc-760cd53e8276 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '523ed589872a45a09b6a7675a7fb0b75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:10.759372 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581b6d42-9ddf-49ab-bffc-760cd53e8276 service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:10.759722 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581b6d42-9ddf-49ab-bffc-760cd53e8276 service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:10.760654 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581b6d42-9ddf-49ab-bffc-760cd53e8276 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:10.762747 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-581b6d42-9ddf-49ab-bffc-760cd53e8276 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:10.869252 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 825/1651] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:10 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:10.883324 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc5763a-5f27-45f4-98df-a5c000da33f8 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:10.885124 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-8cc5763a-5f27-45f4-98df-a5c000da33f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:10.924926 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581b6d42-9ddf-49ab-bffc-760cd53e8276 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:10.926837 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581b6d42-9ddf-49ab-bffc-760cd53e8276 service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:10.966726 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 827/1652] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:31:10 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:31:11.009903 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc5763a-5f27-45f4-98df-a5c000da33f8 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:11.016583 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739c1c15-11a6-4fec-8578-022d3d9f4004 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:11.017402 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739c1c15-11a6-4fec-8578-022d3d9f4004 service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:11.018564 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739c1c15-11a6-4fec-8578-022d3d9f4004 service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:11.018915 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739c1c15-11a6-4fec-8578-022d3d9f4004 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:11.042268 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739c1c15-11a6-4fec-8578-022d3d9f4004 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:11.043838 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739c1c15-11a6-4fec-8578-022d3d9f4004 service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:11.077786 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 826/1653] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:31:10 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:31:11.439659 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581b6d42-9ddf-49ab-bffc-760cd53e8276 service cinder] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:11.439659 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-581b6d42-9ddf-49ab-bffc-760cd53e8276 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:11.456180 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581b6d42-9ddf-49ab-bffc-760cd53e8276 service cinder] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:11.490835 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dac8d7a-2db4-4482-9f3d-f769517ae9cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:11.491546 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dac8d7a-2db4-4482-9f3d-f769517ae9cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:11.491830 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dac8d7a-2db4-4482-9f3d-f769517ae9cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:11.492109 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dac8d7a-2db4-4482-9f3d-f769517ae9cd admin admin] PATH_INFO: `/v3/users/628da841395b4b7e9b3d238c04fe7cee` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:11.493918 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dac8d7a-2db4-4482-9f3d-f769517ae9cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=628da841395b4b7e9b3d238c04fe7cee)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:11.500206 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dac8d7a-2db4-4482-9f3d-f769517ae9cd admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:11.623196 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-7dac8d7a-2db4-4482-9f3d-f769517ae9cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '628da841395b4b7e9b3d238c04fe7cee'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:11.647060 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-7dac8d7a-2db4-4482-9f3d-f769517ae9cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '628da841395b4b7e9b3d238c04fe7cee'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:11.685425 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-7dac8d7a-2db4-4482-9f3d-f769517ae9cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '628da841395b4b7e9b3d238c04fe7cee'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:11.686999 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-7dac8d7a-2db4-4482-9f3d-f769517ae9cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '628da841395b4b7e9b3d238c04fe7cee'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:11.692835 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-7dac8d7a-2db4-4482-9f3d-f769517ae9cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '628da841395b4b7e9b3d238c04fe7cee'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:11.727431 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 828/1654] 10.222.0.102 () {58 vars in 1229 bytes} [Thu Sep 1 17:31:11 2022] DELETE /identity/v3/users/628da841395b4b7e9b3d238c04fe7cee => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:11.732138 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c090d801-0d88-47bf-a09f-66b41392f2d6 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:11.732617 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c090d801-0d88-47bf-a09f-66b41392f2d6 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:11.732690 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c090d801-0d88-47bf-a09f-66b41392f2d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:11.769837 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-c090d801-0d88-47bf-a09f-66b41392f2d6 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:11.773049 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-c090d801-0d88-47bf-a09f-66b41392f2d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:11.923425 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 827/1655] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:11 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:11.975730 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dac8d7a-2db4-4482-9f3d-f769517ae9cd admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:11.977295 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-7dac8d7a-2db4-4482-9f3d-f769517ae9cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:11.992156 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:12.122583 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dac8d7a-2db4-4482-9f3d-f769517ae9cd admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:12.132309 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6b6e7b-c3e6-4af2-a8e0-11d16bb80627 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:12.132633 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6b6e7b-c3e6-4af2-a8e0-11d16bb80627 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:12.132898 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6b6e7b-c3e6-4af2-a8e0-11d16bb80627 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:12.133174 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6b6e7b-c3e6-4af2-a8e0-11d16bb80627 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:12.159556 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6b6e7b-c3e6-4af2-a8e0-11d16bb80627 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:12.161106 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6b6e7b-c3e6-4af2-a8e0-11d16bb80627 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:12.189456 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 829/1656] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:31:11 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:31:13.109360 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c090d801-0d88-47bf-a09f-66b41392f2d6 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:13.111376 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-c090d801-0d88-47bf-a09f-66b41392f2d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:13.153499 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c090d801-0d88-47bf-a09f-66b41392f2d6 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:13.178632 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb8f025-bf95-4d46-a8c5-9accab1ecdea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:13.181350 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb8f025-bf95-4d46-a8c5-9accab1ecdea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:13.181493 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb8f025-bf95-4d46-a8c5-9accab1ecdea admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:13.181779 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb8f025-bf95-4d46-a8c5-9accab1ecdea admin admin] PATH_INFO: `/v3/projects/ad7f8bbb145f4289a09ed9a1bc5a8dfd` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:13.186020 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb8f025-bf95-4d46-a8c5-9accab1ecdea admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad7f8bbb145f4289a09ed9a1bc5a8dfd)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:13.192600 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: WARNING py.warnings [None req-4eb8f025-bf95-4d46-a8c5-9accab1ecdea admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 17:31:13.192600 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: warnings.warn(msg) Sep 01 17:31:13.192600 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: Sep 01 17:31:13.197175 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb8f025-bf95-4d46-a8c5-9accab1ecdea admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:13.268085 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:13.390944 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-4eb8f025-bf95-4d46-a8c5-9accab1ecdea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad7f8bbb145f4289a09ed9a1bc5a8dfd', 'request_id': 'req-4eb8f025-bf95-4d46-a8c5-9accab1ecdea'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:13.411852 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-4eb8f025-bf95-4d46-a8c5-9accab1ecdea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad7f8bbb145f4289a09ed9a1bc5a8dfd', 'request_id': 'req-4eb8f025-bf95-4d46-a8c5-9accab1ecdea'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:13.435860 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-4eb8f025-bf95-4d46-a8c5-9accab1ecdea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad7f8bbb145f4289a09ed9a1bc5a8dfd', 'request_id': 'req-4eb8f025-bf95-4d46-a8c5-9accab1ecdea'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:13.467896 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-4eb8f025-bf95-4d46-a8c5-9accab1ecdea admin admin] The token cache is being invalidate because project ad7f8bbb145f4289a09ed9a1bc5a8dfd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:31:13.468644 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-4eb8f025-bf95-4d46-a8c5-9accab1ecdea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:13.469468 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-4eb8f025-bf95-4d46-a8c5-9accab1ecdea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:13.479178 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6b6e7b-c3e6-4af2-a8e0-11d16bb80627 service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:13.479848 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 828/1657] 10.222.0.102 () {58 vars in 1238 bytes} [Thu Sep 1 17:31:13 2022] DELETE /identity/v3/projects/ad7f8bbb145f4289a09ed9a1bc5a8dfd => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:13.491518 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6b6e7b-c3e6-4af2-a8e0-11d16bb80627 service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:13.591972 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a769e1-5eea-4573-8fd3-70426e95de57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:13.592660 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a769e1-5eea-4573-8fd3-70426e95de57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:13.593002 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a769e1-5eea-4573-8fd3-70426e95de57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:13.593285 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a769e1-5eea-4573-8fd3-70426e95de57 admin admin] PATH_INFO: `/v3/users/2bed612da06848f19a468a8b9aeb3db5` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:13.594568 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a769e1-5eea-4573-8fd3-70426e95de57 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2bed612da06848f19a468a8b9aeb3db5)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:13.596529 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a769e1-5eea-4573-8fd3-70426e95de57 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:13.676502 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-f4a769e1-5eea-4573-8fd3-70426e95de57 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2bed612da06848f19a468a8b9aeb3db5'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:13.713616 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-f4a769e1-5eea-4573-8fd3-70426e95de57 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2bed612da06848f19a468a8b9aeb3db5'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:13.730344 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-f4a769e1-5eea-4573-8fd3-70426e95de57 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2bed612da06848f19a468a8b9aeb3db5'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:13.730996 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-f4a769e1-5eea-4573-8fd3-70426e95de57 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2bed612da06848f19a468a8b9aeb3db5'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:13.731637 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-f4a769e1-5eea-4573-8fd3-70426e95de57 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2bed612da06848f19a468a8b9aeb3db5'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:13.754310 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 830/1658] 10.222.0.102 () {58 vars in 1229 bytes} [Thu Sep 1 17:31:13 2022] DELETE /identity/v3/users/2bed612da06848f19a468a8b9aeb3db5 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:14.826082 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb8f025-bf95-4d46-a8c5-9accab1ecdea admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:14.829099 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-4eb8f025-bf95-4d46-a8c5-9accab1ecdea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:14.883756 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb8f025-bf95-4d46-a8c5-9accab1ecdea admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:15.052733 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ec4850-2827-4e76-abd4-0125296da38a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:15.053481 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ec4850-2827-4e76-abd4-0125296da38a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:15.053779 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ec4850-2827-4e76-abd4-0125296da38a admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:15.054101 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ec4850-2827-4e76-abd4-0125296da38a admin admin] PATH_INFO: `/v3/projects/7864c80b2a9b40aeb18e046fe866cf2b` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:15.060679 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ec4850-2827-4e76-abd4-0125296da38a admin admin] RBAC: Authorizing `identity:delete_project(project_id=7864c80b2a9b40aeb18e046fe866cf2b)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:15.066031 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ec4850-2827-4e76-abd4-0125296da38a admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:15.205235 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-42ec4850-2827-4e76-abd4-0125296da38a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7864c80b2a9b40aeb18e046fe866cf2b', 'request_id': 'req-42ec4850-2827-4e76-abd4-0125296da38a'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:15.219821 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-42ec4850-2827-4e76-abd4-0125296da38a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7864c80b2a9b40aeb18e046fe866cf2b', 'request_id': 'req-42ec4850-2827-4e76-abd4-0125296da38a'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:15.243068 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-42ec4850-2827-4e76-abd4-0125296da38a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7864c80b2a9b40aeb18e046fe866cf2b', 'request_id': 'req-42ec4850-2827-4e76-abd4-0125296da38a'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:15.281251 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-42ec4850-2827-4e76-abd4-0125296da38a admin admin] The token cache is being invalidate because project 7864c80b2a9b40aeb18e046fe866cf2b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:31:15.281585 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-42ec4850-2827-4e76-abd4-0125296da38a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:15.299390 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-42ec4850-2827-4e76-abd4-0125296da38a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:15.305051 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 829/1659] 10.222.0.102 () {58 vars in 1238 bytes} [Thu Sep 1 17:31:14 2022] DELETE /identity/v3/projects/7864c80b2a9b40aeb18e046fe866cf2b => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:15.310540 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca829d2-a6c2-4122-8334-3d5f6c907929 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:15.311162 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca829d2-a6c2-4122-8334-3d5f6c907929 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:15.311162 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca829d2-a6c2-4122-8334-3d5f6c907929 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:15.367410 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-fca829d2-a6c2-4122-8334-3d5f6c907929 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:15.368676 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-fca829d2-a6c2-4122-8334-3d5f6c907929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:15.441390 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:15.595212 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 831/1660] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:15 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:15.603540 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6468d98-a272-43f7-846a-1ed5627fde37 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:15.604264 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6468d98-a272-43f7-846a-1ed5627fde37 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:15.604549 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6468d98-a272-43f7-846a-1ed5627fde37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:15.657526 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-b6468d98-a272-43f7-846a-1ed5627fde37 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:15.658778 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-b6468d98-a272-43f7-846a-1ed5627fde37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:15.685494 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 830/1661] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:15 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:15.688905 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca829d2-a6c2-4122-8334-3d5f6c907929 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:15.704671 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca829d2-a6c2-4122-8334-3d5f6c907929 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:15.720390 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c967733-2286-4d3d-8c5d-61401574490a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:15.721115 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c967733-2286-4d3d-8c5d-61401574490a admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:15.721482 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c967733-2286-4d3d-8c5d-61401574490a admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:15.721708 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c967733-2286-4d3d-8c5d-61401574490a admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:15.723193 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c967733-2286-4d3d-8c5d-61401574490a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:31:15.723193 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c967733-2286-4d3d-8c5d-61401574490a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:15.724568 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c967733-2286-4d3d-8c5d-61401574490a admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:15.775872 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 832/1662] 10.222.0.102 () {58 vars in 1158 bytes} [Thu Sep 1 17:31:15 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:31:15.776167 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6468d98-a272-43f7-846a-1ed5627fde37 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:15.796536 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6468d98-a272-43f7-846a-1ed5627fde37 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:15.817395 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49420681-4a17-4f89-a381-36dd41b8f612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:15.818033 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49420681-4a17-4f89-a381-36dd41b8f612 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:15.818380 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49420681-4a17-4f89-a381-36dd41b8f612 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:15.818573 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49420681-4a17-4f89-a381-36dd41b8f612 admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:15.819240 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49420681-4a17-4f89-a381-36dd41b8f612 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:15.821145 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49420681-4a17-4f89-a381-36dd41b8f612 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:15.886060 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 831/1663] 10.222.0.102 () {60 vars in 1158 bytes} [Thu Sep 1 17:31:15 2022] POST /identity/v3/projects => generated 366 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:31:15.890128 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c967733-2286-4d3d-8c5d-61401574490a admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:15.903841 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c967733-2286-4d3d-8c5d-61401574490a admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:16.017826 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704348d9-c9da-4f7a-b07e-c554a9dfa897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:16.018569 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704348d9-c9da-4f7a-b07e-c554a9dfa897 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:16.018971 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704348d9-c9da-4f7a-b07e-c554a9dfa897 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:16.019363 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704348d9-c9da-4f7a-b07e-c554a9dfa897 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:16.020086 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704348d9-c9da-4f7a-b07e-c554a9dfa897 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:16.023332 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704348d9-c9da-4f7a-b07e-c554a9dfa897 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:16.080255 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 833/1664] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:31:15 2022] POST /identity/v3/users => generated 284 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:31:16.084174 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49420681-4a17-4f89-a381-36dd41b8f612 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:16.099180 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49420681-4a17-4f89-a381-36dd41b8f612 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:16.105203 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1e76fd-8b39-4392-957e-3a383562943a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:16.106021 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1e76fd-8b39-4392-957e-3a383562943a admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:16.106659 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1e76fd-8b39-4392-957e-3a383562943a admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:16.107007 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1e76fd-8b39-4392-957e-3a383562943a admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:16.109051 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1e76fd-8b39-4392-957e-3a383562943a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:16.110878 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1e76fd-8b39-4392-957e-3a383562943a admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:16.115129 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:16.144989 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704348d9-c9da-4f7a-b07e-c554a9dfa897 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:16.145173 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 832/1665] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:31:16 2022] GET /identity/v3/roles => generated 1427 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:31:16.161896 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704348d9-c9da-4f7a-b07e-c554a9dfa897 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:16.186024 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e824445-c7a1-434a-bb02-6caa1968d125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:16.187456 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e824445-c7a1-434a-bb02-6caa1968d125 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:16.187872 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e824445-c7a1-434a-bb02-6caa1968d125 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:16.188376 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e824445-c7a1-434a-bb02-6caa1968d125 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:16.189243 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e824445-c7a1-434a-bb02-6caa1968d125 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:16.192494 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e824445-c7a1-434a-bb02-6caa1968d125 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:16.214405 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1e76fd-8b39-4392-957e-3a383562943a admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:16.224145 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 834/1666] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:31:16 2022] GET /identity/v3/roles => generated 1427 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:31:16.230277 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1e76fd-8b39-4392-957e-3a383562943a admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:16.245671 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236dee8d-9f53-4f71-8810-a39c6062c925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:16.246864 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236dee8d-9f53-4f71-8810-a39c6062c925 admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:16.247199 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236dee8d-9f53-4f71-8810-a39c6062c925 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:16.247635 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236dee8d-9f53-4f71-8810-a39c6062c925 admin admin] PATH_INFO: `/v3/projects/4c09d8bbca464706b2630f2e9917a099/users/b60b45960b354bcaad52bab7ccaf4629/roles/b75cb360925e4ba5a65f14965c103ffd` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:16.282828 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236dee8d-9f53-4f71-8810-a39c6062c925 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c09d8bbca464706b2630f2e9917a099, user_id=b60b45960b354bcaad52bab7ccaf4629, role_id=b75cb360925e4ba5a65f14965c103ffd)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:16.288080 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236dee8d-9f53-4f71-8810-a39c6062c925 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:16.333845 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36c0257-3465-46a6-a4f9-e3bcacbe867d None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:16.337737 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36c0257-3465-46a6-a4f9-e3bcacbe867d None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:16.338050 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36c0257-3465-46a6-a4f9-e3bcacbe867d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:16.338507 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 833/1667] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:31:16 2022] PUT /identity/v3/projects/4c09d8bbca464706b2630f2e9917a099/users/b60b45960b354bcaad52bab7ccaf4629/roles/b75cb360925e4ba5a65f14965c103ffd => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:16.445520 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-f36c0257-3465-46a6-a4f9-e3bcacbe867d None None] MFA Rules not processed for user `b60b45960b354bcaad52bab7ccaf4629`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:16.449052 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-f36c0257-3465-46a6-a4f9-e3bcacbe867d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:16.599365 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236dee8d-9f53-4f71-8810-a39c6062c925 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:16.601692 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-236dee8d-9f53-4f71-8810-a39c6062c925 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:16.729237 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 835/1668] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 401 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:16.730489 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36c0257-3465-46a6-a4f9-e3bcacbe867d None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:16.795196 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:16.845192 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236dee8d-9f53-4f71-8810-a39c6062c925 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:16.861620 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e167a43e-0ba9-4849-b962-ccc28ac78399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:16.862484 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e167a43e-0ba9-4849-b962-ccc28ac78399 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:16.862911 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e167a43e-0ba9-4849-b962-ccc28ac78399 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:16.863629 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e167a43e-0ba9-4849-b962-ccc28ac78399 admin admin] PATH_INFO: `/v3/projects/80bbf3425b2546e48aac5d74cd289cdd` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:16.865462 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e167a43e-0ba9-4849-b962-ccc28ac78399 admin admin] RBAC: Authorizing `identity:delete_project(project_id=80bbf3425b2546e48aac5d74cd289cdd)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:16.870232 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e167a43e-0ba9-4849-b962-ccc28ac78399 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:16.879019 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36c0257-3465-46a6-a4f9-e3bcacbe867d None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:16.898345 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96491102-a71c-453e-9819-71055afadebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:16.899024 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96491102-a71c-453e-9819-71055afadebd admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:16.899329 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96491102-a71c-453e-9819-71055afadebd admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:16.901375 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96491102-a71c-453e-9819-71055afadebd admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:16.902062 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96491102-a71c-453e-9819-71055afadebd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:16.904040 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96491102-a71c-453e-9819-71055afadebd admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:16.956991 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-e167a43e-0ba9-4849-b962-ccc28ac78399 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80bbf3425b2546e48aac5d74cd289cdd', 'request_id': 'req-e167a43e-0ba9-4849-b962-ccc28ac78399'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:16.964808 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-e167a43e-0ba9-4849-b962-ccc28ac78399 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80bbf3425b2546e48aac5d74cd289cdd', 'request_id': 'req-e167a43e-0ba9-4849-b962-ccc28ac78399'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:16.971122 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 836/1669] 10.222.0.102 () {60 vars in 1158 bytes} [Thu Sep 1 17:31:16 2022] POST /identity/v3/projects => generated 366 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:31:16.978710 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96491102-a71c-453e-9819-71055afadebd admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:16.980400 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-e167a43e-0ba9-4849-b962-ccc28ac78399 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80bbf3425b2546e48aac5d74cd289cdd', 'request_id': 'req-e167a43e-0ba9-4849-b962-ccc28ac78399'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:16.995766 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-e167a43e-0ba9-4849-b962-ccc28ac78399 admin admin] The token cache is being invalidate because project 80bbf3425b2546e48aac5d74cd289cdd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:31:16.996387 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-e167a43e-0ba9-4849-b962-ccc28ac78399 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:16.997425 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-e167a43e-0ba9-4849-b962-ccc28ac78399 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:16.999223 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 834/1670] 10.222.0.102 () {58 vars in 1238 bytes} [Thu Sep 1 17:31:16 2022] DELETE /identity/v3/projects/80bbf3425b2546e48aac5d74cd289cdd => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:17.007962 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b531c32a-d270-4f6e-bc6a-c00599df8059 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:17.008501 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b531c32a-d270-4f6e-bc6a-c00599df8059 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:17.008914 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b531c32a-d270-4f6e-bc6a-c00599df8059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:17.055808 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-b531c32a-d270-4f6e-bc6a-c00599df8059 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:17.057066 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-b531c32a-d270-4f6e-bc6a-c00599df8059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:17.137452 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96491102-a71c-453e-9819-71055afadebd admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:17.146546 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b70297a-15bb-4fe6-8c66-e4bcfcc6da84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:17.147399 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b70297a-15bb-4fe6-8c66-e4bcfcc6da84 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:17.147522 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b70297a-15bb-4fe6-8c66-e4bcfcc6da84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:17.147703 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b70297a-15bb-4fe6-8c66-e4bcfcc6da84 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:17.148347 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b70297a-15bb-4fe6-8c66-e4bcfcc6da84 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:17.150354 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b70297a-15bb-4fe6-8c66-e4bcfcc6da84 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:17.187423 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 837/1671] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:31:16 2022] POST /identity/v3/users => generated 290 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:31:17.191288 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b70297a-15bb-4fe6-8c66-e4bcfcc6da84 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:17.195129 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-1b70297a-15bb-4fe6-8c66-e4bcfcc6da84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:17.223667 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b70297a-15bb-4fe6-8c66-e4bcfcc6da84 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:17.237368 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 835/1672] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:17 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:17.242408 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a50000a-7b7e-45dc-afca-49247d88a7bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:17.244572 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a50000a-7b7e-45dc-afca-49247d88a7bc admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:17.244999 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a50000a-7b7e-45dc-afca-49247d88a7bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:17.245384 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9753dcad-7dcd-4764-bf7a-f777e65550a8 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:17.245813 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a50000a-7b7e-45dc-afca-49247d88a7bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:17.246229 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9753dcad-7dcd-4764-bf7a-f777e65550a8 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:17.247127 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a50000a-7b7e-45dc-afca-49247d88a7bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:17.247599 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9753dcad-7dcd-4764-bf7a-f777e65550a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:17.252641 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a50000a-7b7e-45dc-afca-49247d88a7bc admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:17.268545 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:17.281992 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 838/1673] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:31:17 2022] GET /identity/v3/roles => generated 1427 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 17:31:17.282894 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a50000a-7b7e-45dc-afca-49247d88a7bc admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:17.294371 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-9753dcad-7dcd-4764-bf7a-f777e65550a8 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:17.297815 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-9753dcad-7dcd-4764-bf7a-f777e65550a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:17.305353 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a50000a-7b7e-45dc-afca-49247d88a7bc admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:17.323204 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 836/1674] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:17 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:17.329550 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9db1bfc-b146-49d0-bd73-b987373c6c7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:17.331201 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9db1bfc-b146-49d0-bd73-b987373c6c7c admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:17.331488 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9db1bfc-b146-49d0-bd73-b987373c6c7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:17.331714 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9753dcad-7dcd-4764-bf7a-f777e65550a8 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:17.333069 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9db1bfc-b146-49d0-bd73-b987373c6c7c admin admin] PATH_INFO: `/v3/projects/de9eb6f40ecb41f69abb6e4a3711aa0e/users/b6e30767138547f3a0ee83bc76d0ee14/roles/36577a6f46d642d4b81805ede9872490` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:17.345457 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9753dcad-7dcd-4764-bf7a-f777e65550a8 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:17.351699 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9416869f-f120-4b17-a091-77811c7ca5ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:17.352503 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9416869f-f120-4b17-a091-77811c7ca5ef admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:17.352788 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9416869f-f120-4b17-a091-77811c7ca5ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:17.353063 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9416869f-f120-4b17-a091-77811c7ca5ef admin admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:17.353761 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9416869f-f120-4b17-a091-77811c7ca5ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:31:17.354071 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9416869f-f120-4b17-a091-77811c7ca5ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:17.356164 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9416869f-f120-4b17-a091-77811c7ca5ef admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:17.373351 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 837/1675] 10.222.0.102 () {58 vars in 1158 bytes} [Thu Sep 1 17:31:17 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:31:17.376321 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9db1bfc-b146-49d0-bd73-b987373c6c7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=de9eb6f40ecb41f69abb6e4a3711aa0e, user_id=b6e30767138547f3a0ee83bc76d0ee14, role_id=36577a6f46d642d4b81805ede9872490)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:17.379089 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9db1bfc-b146-49d0-bd73-b987373c6c7c admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:17.380572 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9416869f-f120-4b17-a091-77811c7ca5ef admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:17.404004 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9416869f-f120-4b17-a091-77811c7ca5ef admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:17.410554 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 839/1676] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:31:17 2022] PUT /identity/v3/projects/de9eb6f40ecb41f69abb6e4a3711aa0e/users/b6e30767138547f3a0ee83bc76d0ee14/roles/36577a6f46d642d4b81805ede9872490 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:17.415058 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9db1bfc-b146-49d0-bd73-b987373c6c7c admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:17.542621 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca84d7cd-c41c-4439-9f92-5bfb9a10ac2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:17.543392 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca84d7cd-c41c-4439-9f92-5bfb9a10ac2c admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:17.543688 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca84d7cd-c41c-4439-9f92-5bfb9a10ac2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:17.544025 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca84d7cd-c41c-4439-9f92-5bfb9a10ac2c admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:17.544726 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca84d7cd-c41c-4439-9f92-5bfb9a10ac2c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:17.547738 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca84d7cd-c41c-4439-9f92-5bfb9a10ac2c admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:17.552261 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9db1bfc-b146-49d0-bd73-b987373c6c7c admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:17.568608 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb87dacb-82ab-4efa-a670-a02f2a3381cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:17.569436 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb87dacb-82ab-4efa-a670-a02f2a3381cb admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:17.569731 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb87dacb-82ab-4efa-a670-a02f2a3381cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:17.570023 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb87dacb-82ab-4efa-a670-a02f2a3381cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:17.570689 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb87dacb-82ab-4efa-a670-a02f2a3381cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:17.572692 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb87dacb-82ab-4efa-a670-a02f2a3381cb admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:17.586595 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:17.602396 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 840/1677] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:31:17 2022] GET /identity/v3/roles => generated 1427 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:31:17.604635 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb87dacb-82ab-4efa-a670-a02f2a3381cb admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:17.607138 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 838/1678] 10.222.0.102 () {60 vars in 1158 bytes} [Thu Sep 1 17:31:17 2022] POST /identity/v3/projects => generated 388 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:31:17.611716 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca84d7cd-c41c-4439-9f92-5bfb9a10ac2c admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:17.630258 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb87dacb-82ab-4efa-a670-a02f2a3381cb admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:17.775714 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdc98d7-fcf5-4016-84f4-e1203a8b2300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:17.776551 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdc98d7-fcf5-4016-84f4-e1203a8b2300 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:17.776656 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdc98d7-fcf5-4016-84f4-e1203a8b2300 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:17.776869 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdc98d7-fcf5-4016-84f4-e1203a8b2300 admin admin] PATH_INFO: `/v3/projects/de9eb6f40ecb41f69abb6e4a3711aa0e/users/b6e30767138547f3a0ee83bc76d0ee14/roles/36577a6f46d642d4b81805ede9872490` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:17.779594 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdc98d7-fcf5-4016-84f4-e1203a8b2300 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de9eb6f40ecb41f69abb6e4a3711aa0e, user_id=b6e30767138547f3a0ee83bc76d0ee14, role_id=36577a6f46d642d4b81805ede9872490)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:17.785156 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdc98d7-fcf5-4016-84f4-e1203a8b2300 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:17.824066 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 841/1679] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:31:17 2022] PUT /identity/v3/projects/de9eb6f40ecb41f69abb6e4a3711aa0e/users/b6e30767138547f3a0ee83bc76d0ee14/roles/36577a6f46d642d4b81805ede9872490 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:17.826458 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6231173f-0c9a-46ee-8df6-624b2b05a1cd None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:17.826940 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6231173f-0c9a-46ee-8df6-624b2b05a1cd None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:17.827405 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6231173f-0c9a-46ee-8df6-624b2b05a1cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:17.833600 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca84d7cd-c41c-4439-9f92-5bfb9a10ac2c admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:17.875143 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-6231173f-0c9a-46ee-8df6-624b2b05a1cd None None] MFA Rules not processed for user `b6e30767138547f3a0ee83bc76d0ee14`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:17.876399 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-6231173f-0c9a-46ee-8df6-624b2b05a1cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:17.999059 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aec98a-db4a-4f0c-94f1-19c2dc60ed91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:17.999762 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aec98a-db4a-4f0c-94f1-19c2dc60ed91 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:18.000047 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aec98a-db4a-4f0c-94f1-19c2dc60ed91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:18.000504 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aec98a-db4a-4f0c-94f1-19c2dc60ed91 admin admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:18.001199 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28aec98a-db4a-4f0c-94f1-19c2dc60ed91 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:18.003222 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28aec98a-db4a-4f0c-94f1-19c2dc60ed91 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:18.042975 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 842/1680] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:17 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:18.057217 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 839/1681] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:31:17 2022] POST /identity/v3/users => generated 295 bytes in 446 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:31:18.058449 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6231173f-0c9a-46ee-8df6-624b2b05a1cd None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:18.061138 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aec98a-db4a-4f0c-94f1-19c2dc60ed91 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:18.061322 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-6231173f-0c9a-46ee-8df6-624b2b05a1cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:18.071375 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:18.078510 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aec98a-db4a-4f0c-94f1-19c2dc60ed91 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:18.086945 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2864d6-577c-4815-9002-f97ecdd6a664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:18.087745 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2864d6-577c-4815-9002-f97ecdd6a664 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:18.088826 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2864d6-577c-4815-9002-f97ecdd6a664 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:18.089207 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2864d6-577c-4815-9002-f97ecdd6a664 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:18.099638 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2864d6-577c-4815-9002-f97ecdd6a664 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:18.102434 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2864d6-577c-4815-9002-f97ecdd6a664 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:18.126864 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 840/1682] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:31:18 2022] GET /identity/v3/roles => generated 1427 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 17:31:18.137708 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2864d6-577c-4815-9002-f97ecdd6a664 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:18.164041 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2864d6-577c-4815-9002-f97ecdd6a664 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:18.189050 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e58566-a74f-4cac-8ba6-eb9706ee7399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:18.192543 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e58566-a74f-4cac-8ba6-eb9706ee7399 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:18.193200 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e58566-a74f-4cac-8ba6-eb9706ee7399 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:18.193602 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e58566-a74f-4cac-8ba6-eb9706ee7399 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:18.194354 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e58566-a74f-4cac-8ba6-eb9706ee7399 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:18.199100 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6231173f-0c9a-46ee-8df6-624b2b05a1cd None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:18.212462 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e58566-a74f-4cac-8ba6-eb9706ee7399 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:18.214411 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a54cb6-01b1-4e69-ae4f-ac86029ac2a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '523ed589872a45a09b6a7675a7fb0b75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:18.215495 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a54cb6-01b1-4e69-ae4f-ac86029ac2a5 service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:18.215931 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a54cb6-01b1-4e69-ae4f-ac86029ac2a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:18.216920 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a54cb6-01b1-4e69-ae4f-ac86029ac2a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:18.234137 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 841/1683] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:31:18 2022] GET /identity/v3/roles => generated 1427 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:31:18.238621 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e58566-a74f-4cac-8ba6-eb9706ee7399 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:18.241106 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a54cb6-01b1-4e69-ae4f-ac86029ac2a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:18.242644 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a54cb6-01b1-4e69-ae4f-ac86029ac2a5 service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:18.253584 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e58566-a74f-4cac-8ba6-eb9706ee7399 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:18.259007 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d554a6ae-cbf5-4844-8da5-c8d770640134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:18.259917 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d554a6ae-cbf5-4844-8da5-c8d770640134 admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:18.260480 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d554a6ae-cbf5-4844-8da5-c8d770640134 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:18.260945 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d554a6ae-cbf5-4844-8da5-c8d770640134 admin admin] PATH_INFO: `/v3/projects/20168039d6d2498181f81e25b03e0ed2/users/e028a550ab42492b9404b2f28d5ce27b/roles/b75cb360925e4ba5a65f14965c103ffd` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:18.274176 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 843/1684] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:31:18 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:31:18.299375 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d554a6ae-cbf5-4844-8da5-c8d770640134 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20168039d6d2498181f81e25b03e0ed2, user_id=e028a550ab42492b9404b2f28d5ce27b, role_id=b75cb360925e4ba5a65f14965c103ffd)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:18.302370 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d554a6ae-cbf5-4844-8da5-c8d770640134 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:18.340799 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 842/1685] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:31:18 2022] PUT /identity/v3/projects/20168039d6d2498181f81e25b03e0ed2/users/e028a550ab42492b9404b2f28d5ce27b/roles/b75cb360925e4ba5a65f14965c103ffd => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:31:18.358155 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cb6770-0c0b-4862-809a-f02d40a2d535 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:18.359477 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cb6770-0c0b-4862-809a-f02d40a2d535 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:18.359919 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cb6770-0c0b-4862-809a-f02d40a2d535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:18.425258 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-e4cb6770-0c0b-4862-809a-f02d40a2d535 None None] MFA Rules not processed for user `e028a550ab42492b9404b2f28d5ce27b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:18.427748 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-e4cb6770-0c0b-4862-809a-f02d40a2d535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:18.443010 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:18.486778 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7cb431-a9d6-429d-bdd5-907dc0629f2d None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:18.487064 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7cb431-a9d6-429d-bdd5-907dc0629f2d None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:18.487354 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7cb431-a9d6-429d-bdd5-907dc0629f2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:18.562635 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-ee7cb431-a9d6-429d-bdd5-907dc0629f2d None None] MFA Rules not processed for user `26bf17f1fb8948008fed82f4ba33bf66`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:18.564249 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-ee7cb431-a9d6-429d-bdd5-907dc0629f2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:18.582887 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 844/1686] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:18 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:18.587136 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cb6770-0c0b-4862-809a-f02d40a2d535 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:18.704814 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cb6770-0c0b-4862-809a-f02d40a2d535 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:18.715361 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 843/1687] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:18 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:18.721431 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03eae4e-9f1b-4488-b0da-7f952f78d08a None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:18.721722 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03eae4e-9f1b-4488-b0da-7f952f78d08a None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:18.722250 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03eae4e-9f1b-4488-b0da-7f952f78d08a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:18.727631 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab338536-72fc-4ad7-932c-f0beda703db8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:18.731878 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab338536-72fc-4ad7-932c-f0beda703db8 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:18.732180 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab338536-72fc-4ad7-932c-f0beda703db8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:18.734792 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab338536-72fc-4ad7-932c-f0beda703db8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:18.736552 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab338536-72fc-4ad7-932c-f0beda703db8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:18.744612 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab338536-72fc-4ad7-932c-f0beda703db8 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:18.759671 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-c03eae4e-9f1b-4488-b0da-7f952f78d08a None None] MFA Rules not processed for user `d364c74791fc4cd4af6773433a7a13cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:18.771669 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-c03eae4e-9f1b-4488-b0da-7f952f78d08a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:18.822807 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 845/1688] 10.222.0.102 () {60 vars in 1158 bytes} [Thu Sep 1 17:31:18 2022] POST /identity/v3/projects => generated 386 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:31:18.826429 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab338536-72fc-4ad7-932c-f0beda703db8 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:18.832848 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-ab338536-72fc-4ad7-932c-f0beda703db8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:18.980787 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 844/1689] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:18 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:18.981773 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03eae4e-9f1b-4488-b0da-7f952f78d08a None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:19.043667 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03eae4e-9f1b-4488-b0da-7f952f78d08a None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:19.050085 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab338536-72fc-4ad7-932c-f0beda703db8 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:19.066400 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6749e73c-60f9-4c1c-9c67-fe0b4978c19a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f8293316baf4774b62fc11d55d50865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:19.067631 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6749e73c-60f9-4c1c-9c67-fe0b4978c19a service nova] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:19.067923 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6749e73c-60f9-4c1c-9c67-fe0b4978c19a service nova] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:19.068199 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6749e73c-60f9-4c1c-9c67-fe0b4978c19a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:19.124546 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:19.177673 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:19.187194 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6749e73c-60f9-4c1c-9c67-fe0b4978c19a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:19.191183 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6749e73c-60f9-4c1c-9c67-fe0b4978c19a service nova] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:19.224942 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 846/1690] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:31:18 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:31:19.228912 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5140d04-93df-489c-899e-78774e4465a7 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:19.229620 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5140d04-93df-489c-899e-78774e4465a7 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:19.232573 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5140d04-93df-489c-899e-78774e4465a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:19.254920 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84034db-344f-4822-a829-47b5516900c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:19.256245 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84034db-344f-4822-a829-47b5516900c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:19.257825 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84034db-344f-4822-a829-47b5516900c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:19.259031 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84034db-344f-4822-a829-47b5516900c0 admin admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:19.259934 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84034db-344f-4822-a829-47b5516900c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:19.267515 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84034db-344f-4822-a829-47b5516900c0 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:19.296547 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-a5140d04-93df-489c-899e-78774e4465a7 None None] MFA Rules not processed for user `d364c74791fc4cd4af6773433a7a13cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:19.301642 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-a5140d04-93df-489c-899e-78774e4465a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:19.325637 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 845/1691] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:31:18 2022] POST /identity/v3/users => generated 300 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:31:19.326514 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84034db-344f-4822-a829-47b5516900c0 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:19.355086 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84034db-344f-4822-a829-47b5516900c0 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:19.370981 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070041b5-f10e-42a0-b0d1-f18cc0c1a1e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:19.371636 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070041b5-f10e-42a0-b0d1-f18cc0c1a1e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:19.371919 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070041b5-f10e-42a0-b0d1-f18cc0c1a1e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:19.372195 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070041b5-f10e-42a0-b0d1-f18cc0c1a1e5 admin admin] PATH_INFO: `/v3/users/b60b45960b354bcaad52bab7ccaf4629` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:19.374024 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070041b5-f10e-42a0-b0d1-f18cc0c1a1e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b60b45960b354bcaad52bab7ccaf4629)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:19.378392 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070041b5-f10e-42a0-b0d1-f18cc0c1a1e5 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:19.444311 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 847/1692] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:19 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:19.446081 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5140d04-93df-489c-899e-78774e4465a7 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:19.478686 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-070041b5-f10e-42a0-b0d1-f18cc0c1a1e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b60b45960b354bcaad52bab7ccaf4629'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:19.479723 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5140d04-93df-489c-899e-78774e4465a7 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:19.481730 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-070041b5-f10e-42a0-b0d1-f18cc0c1a1e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b60b45960b354bcaad52bab7ccaf4629'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:19.491621 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8259a43-06f7-4335-a54d-9434f3916f5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:19.492262 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8259a43-06f7-4335-a54d-9434f3916f5e admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:19.492524 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8259a43-06f7-4335-a54d-9434f3916f5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:19.492795 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8259a43-06f7-4335-a54d-9434f3916f5e admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:19.493545 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8259a43-06f7-4335-a54d-9434f3916f5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:19.495312 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8259a43-06f7-4335-a54d-9434f3916f5e admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:19.508209 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-070041b5-f10e-42a0-b0d1-f18cc0c1a1e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b60b45960b354bcaad52bab7ccaf4629'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:19.521170 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 848/1693] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:31:19 2022] GET /identity/v3/roles => generated 1427 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:31:19.523405 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8259a43-06f7-4335-a54d-9434f3916f5e admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:19.526537 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-070041b5-f10e-42a0-b0d1-f18cc0c1a1e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b60b45960b354bcaad52bab7ccaf4629'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:19.527233 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-070041b5-f10e-42a0-b0d1-f18cc0c1a1e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b60b45960b354bcaad52bab7ccaf4629'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:19.527336 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-b8259a43-06f7-4335-a54d-9434f3916f5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:19.551160 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 846/1694] 10.222.0.102 () {58 vars in 1229 bytes} [Thu Sep 1 17:31:19 2022] DELETE /identity/v3/users/b60b45960b354bcaad52bab7ccaf4629 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:19.552985 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070041b5-f10e-42a0-b0d1-f18cc0c1a1e5 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:19.563522 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8259a43-06f7-4335-a54d-9434f3916f5e admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:19.569441 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-070041b5-f10e-42a0-b0d1-f18cc0c1a1e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:19.711570 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b2b164-dee4-464b-94ea-2a7767a75417 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f8293316baf4774b62fc11d55d50865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:19.711570 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b2b164-dee4-464b-94ea-2a7767a75417 service nova] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:19.711570 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b2b164-dee4-464b-94ea-2a7767a75417 service nova] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:19.711570 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b2b164-dee4-464b-94ea-2a7767a75417 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:19.713786 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-17b2b164-dee4-464b-94ea-2a7767a75417 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:19.724291 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070041b5-f10e-42a0-b0d1-f18cc0c1a1e5 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:19.732026 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055da524-09a8-400a-a8ee-b8125d72cbda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:19.732725 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055da524-09a8-400a-a8ee-b8125d72cbda admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:19.733003 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055da524-09a8-400a-a8ee-b8125d72cbda admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:19.733357 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055da524-09a8-400a-a8ee-b8125d72cbda admin admin] PATH_INFO: `/v3/projects/41c7ffa7dc1f479789267069d4ac42b5/users/f003069ba7004be6a1cc46065c800036/roles/36577a6f46d642d4b81805ede9872490` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:19.767549 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055da524-09a8-400a-a8ee-b8125d72cbda admin admin] RBAC: Authorizing `identity:create_grant(project_id=41c7ffa7dc1f479789267069d4ac42b5, user_id=f003069ba7004be6a1cc46065c800036, role_id=36577a6f46d642d4b81805ede9872490)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:19.773684 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055da524-09a8-400a-a8ee-b8125d72cbda admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:19.804158 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 847/1695] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:31:19 2022] PUT /identity/v3/projects/41c7ffa7dc1f479789267069d4ac42b5/users/f003069ba7004be6a1cc46065c800036/roles/36577a6f46d642d4b81805ede9872490 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:19.805651 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab479e7-7d51-4016-9e7a-32e9da35cae1 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:19.806127 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab479e7-7d51-4016-9e7a-32e9da35cae1 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:19.806406 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab479e7-7d51-4016-9e7a-32e9da35cae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:19.844735 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b2b164-dee4-464b-94ea-2a7767a75417 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:19.846608 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b2b164-dee4-464b-94ea-2a7767a75417 service nova] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:19.848421 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-0ab479e7-7d51-4016-9e7a-32e9da35cae1 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:19.851944 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-0ab479e7-7d51-4016-9e7a-32e9da35cae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:19.870028 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 849/1696] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:31:19 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:31:19.871075 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b2b164-dee4-464b-94ea-2a7767a75417 service nova] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:19.884676 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:19.892418 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:19.901178 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b2b164-dee4-464b-94ea-2a7767a75417 service nova] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:19.986811 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 848/1697] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:19 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:20.039611 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4b47ee-8fc7-436d-9ea1-a8a664e102cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:20.040834 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4b47ee-8fc7-436d-9ea1-a8a664e102cb admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:20.041416 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4b47ee-8fc7-436d-9ea1-a8a664e102cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:20.041932 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4b47ee-8fc7-436d-9ea1-a8a664e102cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:20.042949 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4b47ee-8fc7-436d-9ea1-a8a664e102cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:20.044976 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4b47ee-8fc7-436d-9ea1-a8a664e102cb admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:20.046651 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab479e7-7d51-4016-9e7a-32e9da35cae1 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:20.048194 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-0ab479e7-7d51-4016-9e7a-32e9da35cae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:20.073797 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 850/1698] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:31:19 2022] GET /identity/v3/roles => generated 1427 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:31:20.078478 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4b47ee-8fc7-436d-9ea1-a8a664e102cb admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:20.097066 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4b47ee-8fc7-436d-9ea1-a8a664e102cb admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:20.102150 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba1f420-7894-41b2-923c-e369a748fa90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:20.102825 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba1f420-7894-41b2-923c-e369a748fa90 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:20.103106 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba1f420-7894-41b2-923c-e369a748fa90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:20.103677 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba1f420-7894-41b2-923c-e369a748fa90 admin admin] PATH_INFO: `/v3/projects/41c7ffa7dc1f479789267069d4ac42b5/users/f003069ba7004be6a1cc46065c800036/roles/36577a6f46d642d4b81805ede9872490` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:20.106285 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba1f420-7894-41b2-923c-e369a748fa90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41c7ffa7dc1f479789267069d4ac42b5, user_id=f003069ba7004be6a1cc46065c800036, role_id=36577a6f46d642d4b81805ede9872490)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:20.109289 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba1f420-7894-41b2-923c-e369a748fa90 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:20.123236 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 851/1699] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:31:20 2022] PUT /identity/v3/projects/41c7ffa7dc1f479789267069d4ac42b5/users/f003069ba7004be6a1cc46065c800036/roles/36577a6f46d642d4b81805ede9872490 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:20.136699 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10aae46-bf69-476e-b7e1-ae7edd8a2b3f None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:20.137104 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10aae46-bf69-476e-b7e1-ae7edd8a2b3f None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:20.138019 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10aae46-bf69-476e-b7e1-ae7edd8a2b3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:20.158186 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab479e7-7d51-4016-9e7a-32e9da35cae1 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:20.183962 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-c10aae46-bf69-476e-b7e1-ae7edd8a2b3f None None] MFA Rules not processed for user `f003069ba7004be6a1cc46065c800036`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:20.187514 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-c10aae46-bf69-476e-b7e1-ae7edd8a2b3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:20.273032 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1c349d-2f55-460f-ae54-70b40a850ee6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:20.274109 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1c349d-2f55-460f-ae54-70b40a850ee6 service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:20.274559 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1c349d-2f55-460f-ae54-70b40a850ee6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:20.274993 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1c349d-2f55-460f-ae54-70b40a850ee6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:20.368165 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 852/1700] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:20 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:20.384223 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10aae46-bf69-476e-b7e1-ae7edd8a2b3f None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:20.389218 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-c10aae46-bf69-476e-b7e1-ae7edd8a2b3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:20.421013 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1c349d-2f55-460f-ae54-70b40a850ee6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:20.424994 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1c349d-2f55-460f-ae54-70b40a850ee6 service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:20.450301 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 849/1701] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:31:20 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 17:31:20.495747 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10aae46-bf69-476e-b7e1-ae7edd8a2b3f None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:20.505803 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace9a580-9b99-4c7f-8e6c-9e424f1463ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '523ed589872a45a09b6a7675a7fb0b75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:20.506516 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace9a580-9b99-4c7f-8e6c-9e424f1463ef service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:20.506845 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace9a580-9b99-4c7f-8e6c-9e424f1463ef service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:20.507173 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace9a580-9b99-4c7f-8e6c-9e424f1463ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:20.540502 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace9a580-9b99-4c7f-8e6c-9e424f1463ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:20.542080 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace9a580-9b99-4c7f-8e6c-9e424f1463ef service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:20.560761 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 853/1702] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:31:20 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:31:20.693183 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1c349d-2f55-460f-ae54-70b40a850ee6 service neutron] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:20.706973 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1c349d-2f55-460f-ae54-70b40a850ee6 service neutron] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:20.720282 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89624be8-14d3-40f2-86bd-2d5f39f3378d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:20.720940 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89624be8-14d3-40f2-86bd-2d5f39f3378d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:20.721214 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89624be8-14d3-40f2-86bd-2d5f39f3378d admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:20.721511 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89624be8-14d3-40f2-86bd-2d5f39f3378d admin admin] PATH_INFO: `/v3/users/e028a550ab42492b9404b2f28d5ce27b` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:20.726069 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89624be8-14d3-40f2-86bd-2d5f39f3378d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e028a550ab42492b9404b2f28d5ce27b)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:20.728231 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89624be8-14d3-40f2-86bd-2d5f39f3378d admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:20.777240 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:20.831950 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-89624be8-14d3-40f2-86bd-2d5f39f3378d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e028a550ab42492b9404b2f28d5ce27b'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:20.832683 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-89624be8-14d3-40f2-86bd-2d5f39f3378d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e028a550ab42492b9404b2f28d5ce27b'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:20.861901 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-89624be8-14d3-40f2-86bd-2d5f39f3378d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e028a550ab42492b9404b2f28d5ce27b'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:20.885990 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-89624be8-14d3-40f2-86bd-2d5f39f3378d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e028a550ab42492b9404b2f28d5ce27b'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:20.886634 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-89624be8-14d3-40f2-86bd-2d5f39f3378d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e028a550ab42492b9404b2f28d5ce27b'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:20.906745 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 850/1703] 10.222.0.102 () {58 vars in 1229 bytes} [Thu Sep 1 17:31:20 2022] DELETE /identity/v3/users/e028a550ab42492b9404b2f28d5ce27b => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:20.914264 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176b4de5-23ee-4c7f-9109-5cef204df2db None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:20.914264 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176b4de5-23ee-4c7f-9109-5cef204df2db None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:20.914728 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176b4de5-23ee-4c7f-9109-5cef204df2db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:20.971296 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-176b4de5-23ee-4c7f-9109-5cef204df2db None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:20.972817 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-176b4de5-23ee-4c7f-9109-5cef204df2db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:20.993407 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:21.113263 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 854/1704] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:20 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:21.138690 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89624be8-14d3-40f2-86bd-2d5f39f3378d admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:21.142235 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-89624be8-14d3-40f2-86bd-2d5f39f3378d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:21.164241 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c97cf70-ab5d-4acc-8901-d7267fec2947 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:21.165937 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c97cf70-ab5d-4acc-8901-d7267fec2947 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:21.168524 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c97cf70-ab5d-4acc-8901-d7267fec2947 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:21.180911 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 855/1705] 10.222.0.102 () {54 vars in 917 bytes} [Thu Sep 1 17:31:21 2022] GET /identity => generated 271 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:31:21.187120 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347df6ca-7960-4654-903e-2d8a662a0ad9 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:21.187410 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347df6ca-7960-4654-903e-2d8a662a0ad9 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:21.187670 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347df6ca-7960-4654-903e-2d8a662a0ad9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:21.265234 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89624be8-14d3-40f2-86bd-2d5f39f3378d admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:21.277084 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a319908-bab2-4f2e-8ed1-4180d717faf6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:21.277916 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a319908-bab2-4f2e-8ed1-4180d717faf6 service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:21.278094 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a319908-bab2-4f2e-8ed1-4180d717faf6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:21.278370 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a319908-bab2-4f2e-8ed1-4180d717faf6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:21.279746 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-347df6ca-7960-4654-903e-2d8a662a0ad9 None None] MFA Rules not processed for user `7c7d3594fdab4191be1d841af43a5c79`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:21.283341 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-347df6ca-7960-4654-903e-2d8a662a0ad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:21.298136 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a319908-bab2-4f2e-8ed1-4180d717faf6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:21.299626 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a319908-bab2-4f2e-8ed1-4180d717faf6 service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:21.300451 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 856/1706] 10.222.0.102 () {58 vars in 1014 bytes} [Thu Sep 1 17:31:21 2022] POST /identity/v3/auth/tokens => generated 3131 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:31:21.332847 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 851/1707] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:31:21 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:31:21.389187 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347df6ca-7960-4654-903e-2d8a662a0ad9 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:21.422234 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347df6ca-7960-4654-903e-2d8a662a0ad9 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:21.426277 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a1a80b-9492-4f33-8a85-4d6968178910 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:21.426904 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a1a80b-9492-4f33-8a85-4d6968178910 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:21.427192 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a1a80b-9492-4f33-8a85-4d6968178910 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:21.427469 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a1a80b-9492-4f33-8a85-4d6968178910 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:21.441246 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a1a80b-9492-4f33-8a85-4d6968178910 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:21.460158 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a1a80b-9492-4f33-8a85-4d6968178910 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:21.489093 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 857/1708] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:31:21 2022] GET /identity/v3/auth/tokens => generated 3131 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:31:21.878010 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a319908-bab2-4f2e-8ed1-4180d717faf6 service neutron] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:21.880016 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-2a319908-bab2-4f2e-8ed1-4180d717faf6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:21.932011 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a319908-bab2-4f2e-8ed1-4180d717faf6 service neutron] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:21.962308 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d34761f-6579-4294-bdf2-d7809bac809e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:21.962977 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d34761f-6579-4294-bdf2-d7809bac809e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:21.963265 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d34761f-6579-4294-bdf2-d7809bac809e admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:21.966245 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d34761f-6579-4294-bdf2-d7809bac809e admin admin] PATH_INFO: `/v3/projects/4c09d8bbca464706b2630f2e9917a099` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:21.970119 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d34761f-6579-4294-bdf2-d7809bac809e admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c09d8bbca464706b2630f2e9917a099)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:21.975674 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d34761f-6579-4294-bdf2-d7809bac809e admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:22.156239 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-5d34761f-6579-4294-bdf2-d7809bac809e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c09d8bbca464706b2630f2e9917a099', 'request_id': 'req-5d34761f-6579-4294-bdf2-d7809bac809e'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:22.179991 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-5d34761f-6579-4294-bdf2-d7809bac809e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c09d8bbca464706b2630f2e9917a099', 'request_id': 'req-5d34761f-6579-4294-bdf2-d7809bac809e'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:22.209060 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-5d34761f-6579-4294-bdf2-d7809bac809e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c09d8bbca464706b2630f2e9917a099', 'request_id': 'req-5d34761f-6579-4294-bdf2-d7809bac809e'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:22.242320 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-5d34761f-6579-4294-bdf2-d7809bac809e admin admin] The token cache is being invalidate because project 4c09d8bbca464706b2630f2e9917a099 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:31:22.242712 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-5d34761f-6579-4294-bdf2-d7809bac809e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:22.244051 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-5d34761f-6579-4294-bdf2-d7809bac809e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:22.251958 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 852/1709] 10.222.0.102 () {58 vars in 1238 bytes} [Thu Sep 1 17:31:21 2022] DELETE /identity/v3/projects/4c09d8bbca464706b2630f2e9917a099 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:22.256168 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a1a80b-9492-4f33-8a85-4d6968178910 service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:22.276645 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a1a80b-9492-4f33-8a85-4d6968178910 service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:22.392397 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4547b0e8-e93a-4181-8c1b-c6414df6dd50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:22.396032 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4547b0e8-e93a-4181-8c1b-c6414df6dd50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:22.396514 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4547b0e8-e93a-4181-8c1b-c6414df6dd50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:22.398338 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4547b0e8-e93a-4181-8c1b-c6414df6dd50 admin admin] PATH_INFO: `/v3/users/b6e30767138547f3a0ee83bc76d0ee14` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:22.402082 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4547b0e8-e93a-4181-8c1b-c6414df6dd50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6e30767138547f3a0ee83bc76d0ee14)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:22.408069 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4547b0e8-e93a-4181-8c1b-c6414df6dd50 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:22.540709 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-4547b0e8-e93a-4181-8c1b-c6414df6dd50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6e30767138547f3a0ee83bc76d0ee14'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:22.571826 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-4547b0e8-e93a-4181-8c1b-c6414df6dd50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6e30767138547f3a0ee83bc76d0ee14'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:22.592603 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-4547b0e8-e93a-4181-8c1b-c6414df6dd50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6e30767138547f3a0ee83bc76d0ee14'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:22.593335 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-4547b0e8-e93a-4181-8c1b-c6414df6dd50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6e30767138547f3a0ee83bc76d0ee14'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:22.593995 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-4547b0e8-e93a-4181-8c1b-c6414df6dd50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6e30767138547f3a0ee83bc76d0ee14'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:22.621262 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 858/1710] 10.222.0.102 () {58 vars in 1229 bytes} [Thu Sep 1 17:31:22 2022] DELETE /identity/v3/users/b6e30767138547f3a0ee83bc76d0ee14 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:23.150923 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d34761f-6579-4294-bdf2-d7809bac809e admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:23.153609 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4547b0e8-e93a-4181-8c1b-c6414df6dd50 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:23.155710 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-5d34761f-6579-4294-bdf2-d7809bac809e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:23.157595 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-4547b0e8-e93a-4181-8c1b-c6414df6dd50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:23.164621 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:23.238174 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:23.288483 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4547b0e8-e93a-4181-8c1b-c6414df6dd50 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:23.327897 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2828c067-eb10-4a80-a1b3-a06aaa51a761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:23.328569 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2828c067-eb10-4a80-a1b3-a06aaa51a761 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:23.328840 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2828c067-eb10-4a80-a1b3-a06aaa51a761 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:23.329119 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2828c067-eb10-4a80-a1b3-a06aaa51a761 admin admin] PATH_INFO: `/v3/projects/20168039d6d2498181f81e25b03e0ed2` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:23.330311 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2828c067-eb10-4a80-a1b3-a06aaa51a761 admin admin] RBAC: Authorizing `identity:delete_project(project_id=20168039d6d2498181f81e25b03e0ed2)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:23.333337 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2828c067-eb10-4a80-a1b3-a06aaa51a761 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:23.362534 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d34761f-6579-4294-bdf2-d7809bac809e admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:23.456940 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-2828c067-eb10-4a80-a1b3-a06aaa51a761 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20168039d6d2498181f81e25b03e0ed2', 'request_id': 'req-2828c067-eb10-4a80-a1b3-a06aaa51a761'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:23.470410 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-2828c067-eb10-4a80-a1b3-a06aaa51a761 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20168039d6d2498181f81e25b03e0ed2', 'request_id': 'req-2828c067-eb10-4a80-a1b3-a06aaa51a761'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:23.489849 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-2828c067-eb10-4a80-a1b3-a06aaa51a761 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20168039d6d2498181f81e25b03e0ed2', 'request_id': 'req-2828c067-eb10-4a80-a1b3-a06aaa51a761'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:23.514665 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-2828c067-eb10-4a80-a1b3-a06aaa51a761 admin admin] The token cache is being invalidate because project 20168039d6d2498181f81e25b03e0ed2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:31:23.514972 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-2828c067-eb10-4a80-a1b3-a06aaa51a761 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:23.517646 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-2828c067-eb10-4a80-a1b3-a06aaa51a761 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:23.521630 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 859/1711] 10.222.0.102 () {58 vars in 1238 bytes} [Thu Sep 1 17:31:23 2022] DELETE /identity/v3/projects/20168039d6d2498181f81e25b03e0ed2 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:23.523102 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba8096a-3bb5-47ee-a124-76be1ca46d1c None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:23.523399 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba8096a-3bb5-47ee-a124-76be1ca46d1c None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:23.525659 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba8096a-3bb5-47ee-a124-76be1ca46d1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:23.560652 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-9ba8096a-3bb5-47ee-a124-76be1ca46d1c None None] MFA Rules not processed for user `26bf17f1fb8948008fed82f4ba33bf66`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:23.562289 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-9ba8096a-3bb5-47ee-a124-76be1ca46d1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:23.631567 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5478c287-6f38-49dd-b196-d8615b9781bb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '134ef6039cef468bae0e3f3be79b8073', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:23.632220 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5478c287-6f38-49dd-b196-d8615b9781bb service placement] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:23.632506 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5478c287-6f38-49dd-b196-d8615b9781bb service placement] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:23.632782 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5478c287-6f38-49dd-b196-d8615b9781bb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:23.647107 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-5478c287-6f38-49dd-b196-d8615b9781bb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:23.660955 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 860/1712] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:23 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:23.662803 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a67ea53-e6ef-4c9e-8c75-32a54dfe6441 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:23.663101 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a67ea53-e6ef-4c9e-8c75-32a54dfe6441 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:23.663381 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a67ea53-e6ef-4c9e-8c75-32a54dfe6441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:23.684874 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5478c287-6f38-49dd-b196-d8615b9781bb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:23.686098 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-0a67ea53-e6ef-4c9e-8c75-32a54dfe6441 None None] MFA Rules not processed for user `d364c74791fc4cd4af6773433a7a13cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:23.686696 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5478c287-6f38-49dd-b196-d8615b9781bb service placement] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:23.687355 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-0a67ea53-e6ef-4c9e-8c75-32a54dfe6441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:23.716990 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 853/1713] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:31:23 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 566 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:31:23.742360 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5478c287-6f38-49dd-b196-d8615b9781bb service placement] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:23.747076 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-5478c287-6f38-49dd-b196-d8615b9781bb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:23.755844 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:23.823578 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 861/1714] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:23 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:23.845222 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5478c287-6f38-49dd-b196-d8615b9781bb service placement] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:23.951301 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28094906-6870-4f00-87a7-8355cf81b622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:23.952667 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28094906-6870-4f00-87a7-8355cf81b622 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:23.952956 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28094906-6870-4f00-87a7-8355cf81b622 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:23.953232 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28094906-6870-4f00-87a7-8355cf81b622 admin admin] PATH_INFO: `/v3/users/f003069ba7004be6a1cc46065c800036` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:23.957362 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28094906-6870-4f00-87a7-8355cf81b622 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f003069ba7004be6a1cc46065c800036)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:23.959623 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28094906-6870-4f00-87a7-8355cf81b622 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:23.981280 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a67ea53-e6ef-4c9e-8c75-32a54dfe6441 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:23.983831 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-0a67ea53-e6ef-4c9e-8c75-32a54dfe6441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:24.030299 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a67ea53-e6ef-4c9e-8c75-32a54dfe6441 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:24.046321 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff618547-10b6-4128-8737-2a6c00ca8561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:24.050220 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff618547-10b6-4128-8737-2a6c00ca8561 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:24.050805 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff618547-10b6-4128-8737-2a6c00ca8561 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:24.051136 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff618547-10b6-4128-8737-2a6c00ca8561 admin admin] PATH_INFO: `/v3/projects/de9eb6f40ecb41f69abb6e4a3711aa0e` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:24.055434 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff618547-10b6-4128-8737-2a6c00ca8561 admin admin] RBAC: Authorizing `identity:delete_project(project_id=de9eb6f40ecb41f69abb6e4a3711aa0e)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:24.060530 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff618547-10b6-4128-8737-2a6c00ca8561 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:24.094034 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-28094906-6870-4f00-87a7-8355cf81b622 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f003069ba7004be6a1cc46065c800036'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:24.094854 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-28094906-6870-4f00-87a7-8355cf81b622 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f003069ba7004be6a1cc46065c800036'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:24.116186 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:24.121865 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-28094906-6870-4f00-87a7-8355cf81b622 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f003069ba7004be6a1cc46065c800036'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:24.142644 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-28094906-6870-4f00-87a7-8355cf81b622 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f003069ba7004be6a1cc46065c800036'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:24.143400 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-28094906-6870-4f00-87a7-8355cf81b622 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f003069ba7004be6a1cc46065c800036'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:24.167387 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 854/1715] 10.222.0.102 () {58 vars in 1229 bytes} [Thu Sep 1 17:31:23 2022] DELETE /identity/v3/users/f003069ba7004be6a1cc46065c800036 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:24.213016 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-ff618547-10b6-4128-8737-2a6c00ca8561 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de9eb6f40ecb41f69abb6e4a3711aa0e', 'request_id': 'req-ff618547-10b6-4128-8737-2a6c00ca8561'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:24.230343 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-ff618547-10b6-4128-8737-2a6c00ca8561 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de9eb6f40ecb41f69abb6e4a3711aa0e', 'request_id': 'req-ff618547-10b6-4128-8737-2a6c00ca8561'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:24.242345 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-ff618547-10b6-4128-8737-2a6c00ca8561 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de9eb6f40ecb41f69abb6e4a3711aa0e', 'request_id': 'req-ff618547-10b6-4128-8737-2a6c00ca8561'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:24.275156 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-ff618547-10b6-4128-8737-2a6c00ca8561 admin admin] The token cache is being invalidate because project de9eb6f40ecb41f69abb6e4a3711aa0e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:31:24.275530 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-ff618547-10b6-4128-8737-2a6c00ca8561 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:24.278922 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-ff618547-10b6-4128-8737-2a6c00ca8561 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:24.285686 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 862/1716] 10.222.0.102 () {58 vars in 1238 bytes} [Thu Sep 1 17:31:23 2022] DELETE /identity/v3/projects/de9eb6f40ecb41f69abb6e4a3711aa0e => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:31:24.290387 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0c37a7-bc0a-44a4-860c-08dac95a2b2d None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:24.290818 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0c37a7-bc0a-44a4-860c-08dac95a2b2d None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:24.290991 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0c37a7-bc0a-44a4-860c-08dac95a2b2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:24.337341 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-9c0c37a7-bc0a-44a4-860c-08dac95a2b2d None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:24.344737 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-9c0c37a7-bc0a-44a4-860c-08dac95a2b2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:24.460955 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9d8c80-cc44-429b-aa82-a8262de4665c None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:24.461825 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9d8c80-cc44-429b-aa82-a8262de4665c None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:24.462298 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9d8c80-cc44-429b-aa82-a8262de4665c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:24.527862 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 855/1717] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:24 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:24.536227 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6046843-b1e2-4b77-905e-0bc1726fd4b1 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:24.536565 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6046843-b1e2-4b77-905e-0bc1726fd4b1 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:24.536831 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6046843-b1e2-4b77-905e-0bc1726fd4b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:24.570501 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-1e9d8c80-cc44-429b-aa82-a8262de4665c None None] MFA Rules not processed for user `19a53d1e1a84408ea1dc2aba24bd8299`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:24.574471 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-1e9d8c80-cc44-429b-aa82-a8262de4665c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:24.595271 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-f6046843-b1e2-4b77-905e-0bc1726fd4b1 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:24.598700 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-f6046843-b1e2-4b77-905e-0bc1726fd4b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:24.627200 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 856/1718] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:24 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:24.633349 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6046843-b1e2-4b77-905e-0bc1726fd4b1 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:24.674916 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6046843-b1e2-4b77-905e-0bc1726fd4b1 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:24.693837 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208fc610-e60f-487c-bdd6-0abc3903a1d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:24.694613 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208fc610-e60f-487c-bdd6-0abc3903a1d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:24.694989 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208fc610-e60f-487c-bdd6-0abc3903a1d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:24.695360 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208fc610-e60f-487c-bdd6-0abc3903a1d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:24.696177 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208fc610-e60f-487c-bdd6-0abc3903a1d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:31:24.696774 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208fc610-e60f-487c-bdd6-0abc3903a1d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:24.698635 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208fc610-e60f-487c-bdd6-0abc3903a1d3 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:24.727758 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 857/1719] 10.222.0.102 () {58 vars in 1158 bytes} [Thu Sep 1 17:31:24 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:31:24.729364 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208fc610-e60f-487c-bdd6-0abc3903a1d3 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:24.753741 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208fc610-e60f-487c-bdd6-0abc3903a1d3 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:24.763784 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 863/1720] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:24 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:24.772040 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da64dcb-48a3-4338-8335-a5910fc7372b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:24.774076 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da64dcb-48a3-4338-8335-a5910fc7372b admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:24.775558 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da64dcb-48a3-4338-8335-a5910fc7372b admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:24.775941 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da64dcb-48a3-4338-8335-a5910fc7372b admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:24.777253 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d328d4c5-2b86-4b41-9e99-3b6780ee7007 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:24.777731 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d328d4c5-2b86-4b41-9e99-3b6780ee7007 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:24.778154 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d328d4c5-2b86-4b41-9e99-3b6780ee7007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:24.778367 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da64dcb-48a3-4338-8335-a5910fc7372b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:24.782867 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da64dcb-48a3-4338-8335-a5910fc7372b admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:24.852934 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-d328d4c5-2b86-4b41-9e99-3b6780ee7007 None None] MFA Rules not processed for user `19a53d1e1a84408ea1dc2aba24bd8299`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:24.852934 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-d328d4c5-2b86-4b41-9e99-3b6780ee7007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:24.853162 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 858/1721] 10.222.0.102 () {60 vars in 1158 bytes} [Thu Sep 1 17:31:24 2022] POST /identity/v3/projects => generated 408 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:31:24.854263 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da64dcb-48a3-4338-8335-a5910fc7372b admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:24.993475 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:25.017356 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 864/1722] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:24 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:25.018897 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3cb4bd-69db-4057-ae89-68b0d3f5d275 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:25.018897 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3cb4bd-69db-4057-ae89-68b0d3f5d275 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:25.018897 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3cb4bd-69db-4057-ae89-68b0d3f5d275 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:25.022741 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 865/1723] 10.222.0.102 () {54 vars in 917 bytes} [Thu Sep 1 17:31:25 2022] GET /identity => generated 271 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:31:25.035107 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4f88c1-8b7d-4fff-be7f-dc8e0a3506bf None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:25.035482 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4f88c1-8b7d-4fff-be7f-dc8e0a3506bf None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:25.035676 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4f88c1-8b7d-4fff-be7f-dc8e0a3506bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:25.051357 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da64dcb-48a3-4338-8335-a5910fc7372b admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:25.058286 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eae06ab-84d3-4956-85f8-c08851fba378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:25.059204 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eae06ab-84d3-4956-85f8-c08851fba378 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:25.059541 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eae06ab-84d3-4956-85f8-c08851fba378 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:25.060005 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eae06ab-84d3-4956-85f8-c08851fba378 admin admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:25.060762 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eae06ab-84d3-4956-85f8-c08851fba378 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:25.063065 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eae06ab-84d3-4956-85f8-c08851fba378 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:25.104908 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 859/1724] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:31:24 2022] POST /identity/v3/users => generated 305 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:31:25.117719 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eae06ab-84d3-4956-85f8-c08851fba378 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:25.117797 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-5e4f88c1-8b7d-4fff-be7f-dc8e0a3506bf None None] MFA Rules not processed for user `7c7d3594fdab4191be1d841af43a5c79`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:25.118045 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-5e4f88c1-8b7d-4fff-be7f-dc8e0a3506bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:25.140928 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eae06ab-84d3-4956-85f8-c08851fba378 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:25.151523 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c837005-7374-4aa0-9608-00f5c87e1138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:25.154604 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c837005-7374-4aa0-9608-00f5c87e1138 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:25.154604 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c837005-7374-4aa0-9608-00f5c87e1138 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:25.154842 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c837005-7374-4aa0-9608-00f5c87e1138 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:25.155994 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c837005-7374-4aa0-9608-00f5c87e1138 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:25.157616 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c837005-7374-4aa0-9608-00f5c87e1138 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:25.185847 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 860/1725] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:31:25 2022] GET /identity/v3/roles => generated 1427 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:31:25.187203 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c837005-7374-4aa0-9608-00f5c87e1138 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:25.201263 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c837005-7374-4aa0-9608-00f5c87e1138 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:25.212544 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da17a99b-8084-473a-9475-e5671073cb3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:25.213589 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da17a99b-8084-473a-9475-e5671073cb3f admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:25.213870 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da17a99b-8084-473a-9475-e5671073cb3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:25.214143 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da17a99b-8084-473a-9475-e5671073cb3f admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:25.214788 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da17a99b-8084-473a-9475-e5671073cb3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:25.216833 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da17a99b-8084-473a-9475-e5671073cb3f admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:25.233425 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 861/1726] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:31:25 2022] GET /identity/v3/roles => generated 1427 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:31:25.237382 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da17a99b-8084-473a-9475-e5671073cb3f admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:25.245631 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:DELETE=2,keystone:SELECT=93,keystone:INSERT=5 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:25.246928 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 866/1727] 10.222.0.102 () {58 vars in 1014 bytes} [Thu Sep 1 17:31:25 2022] POST /identity/v3/auth/tokens => generated 3131 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:25.250600 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da17a99b-8084-473a-9475-e5671073cb3f admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:25.258387 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a360f14-5668-4cc8-87b6-b186b705c9bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:25.259404 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a360f14-5668-4cc8-87b6-b186b705c9bf admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:25.259778 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a360f14-5668-4cc8-87b6-b186b705c9bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:25.260118 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a360f14-5668-4cc8-87b6-b186b705c9bf admin admin] PATH_INFO: `/v3/projects/322855688b6f482495822cf7b84460b2/users/8803299d636346c69c74eb1498d6f267/roles/b75cb360925e4ba5a65f14965c103ffd` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:25.280924 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a360f14-5668-4cc8-87b6-b186b705c9bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=322855688b6f482495822cf7b84460b2, user_id=8803299d636346c69c74eb1498d6f267, role_id=b75cb360925e4ba5a65f14965c103ffd)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:25.282377 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4f88c1-8b7d-4fff-be7f-dc8e0a3506bf None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:25.283868 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-5e4f88c1-8b7d-4fff-be7f-dc8e0a3506bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:25.285092 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a360f14-5668-4cc8-87b6-b186b705c9bf admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:25.304548 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4f88c1-8b7d-4fff-be7f-dc8e0a3506bf None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:25.309733 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 862/1728] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:31:25 2022] PUT /identity/v3/projects/322855688b6f482495822cf7b84460b2/users/8803299d636346c69c74eb1498d6f267/roles/b75cb360925e4ba5a65f14965c103ffd => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:31:25.318784 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094596e0-5972-462e-9a3e-c8b3b6556b2f None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:25.319432 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094596e0-5972-462e-9a3e-c8b3b6556b2f None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:25.320002 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094596e0-5972-462e-9a3e-c8b3b6556b2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:25.365204 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-094596e0-5972-462e-9a3e-c8b3b6556b2f None None] MFA Rules not processed for user `8803299d636346c69c74eb1498d6f267`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:25.368312 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-094596e0-5972-462e-9a3e-c8b3b6556b2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:25.417630 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812c7164-9c1a-4d65-afbf-237d39c38275 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:25.418254 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812c7164-9c1a-4d65-afbf-237d39c38275 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:25.418538 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812c7164-9c1a-4d65-afbf-237d39c38275 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:25.418813 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812c7164-9c1a-4d65-afbf-237d39c38275 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:25.437251 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812c7164-9c1a-4d65-afbf-237d39c38275 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:25.438826 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812c7164-9c1a-4d65-afbf-237d39c38275 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:25.447691 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 867/1729] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:31:25 2022] GET /identity/v3/auth/tokens => generated 3131 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 17:31:25.448616 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812c7164-9c1a-4d65-afbf-237d39c38275 service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:25.450082 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-812c7164-9c1a-4d65-afbf-237d39c38275 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:25.496349 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 863/1730] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:25 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:25.501517 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094596e0-5972-462e-9a3e-c8b3b6556b2f None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:25.544643 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812c7164-9c1a-4d65-afbf-237d39c38275 service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:25.549971 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214e0c5c-a125-4531-9bc4-b103829b76b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:25.550735 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214e0c5c-a125-4531-9bc4-b103829b76b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:25.551098 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214e0c5c-a125-4531-9bc4-b103829b76b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:25.551645 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214e0c5c-a125-4531-9bc4-b103829b76b8 admin admin] PATH_INFO: `/v3/projects/41c7ffa7dc1f479789267069d4ac42b5` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:25.553768 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214e0c5c-a125-4531-9bc4-b103829b76b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=41c7ffa7dc1f479789267069d4ac42b5)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:25.555802 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214e0c5c-a125-4531-9bc4-b103829b76b8 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:25.651629 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-214e0c5c-a125-4531-9bc4-b103829b76b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41c7ffa7dc1f479789267069d4ac42b5', 'request_id': 'req-214e0c5c-a125-4531-9bc4-b103829b76b8'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:25.652912 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094596e0-5972-462e-9a3e-c8b3b6556b2f None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:25.658148 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:25.665538 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-214e0c5c-a125-4531-9bc4-b103829b76b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41c7ffa7dc1f479789267069d4ac42b5', 'request_id': 'req-214e0c5c-a125-4531-9bc4-b103829b76b8'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:25.675968 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-214e0c5c-a125-4531-9bc4-b103829b76b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41c7ffa7dc1f479789267069d4ac42b5', 'request_id': 'req-214e0c5c-a125-4531-9bc4-b103829b76b8'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:25.699590 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-214e0c5c-a125-4531-9bc4-b103829b76b8 admin admin] The token cache is being invalidate because project 41c7ffa7dc1f479789267069d4ac42b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:31:25.700070 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-214e0c5c-a125-4531-9bc4-b103829b76b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:25.701444 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-214e0c5c-a125-4531-9bc4-b103829b76b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:25.703935 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 868/1731] 10.222.0.102 () {58 vars in 1238 bytes} [Thu Sep 1 17:31:25 2022] DELETE /identity/v3/projects/41c7ffa7dc1f479789267069d4ac42b5 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:25.707276 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214e0c5c-a125-4531-9bc4-b103829b76b8 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:25.708304 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-214e0c5c-a125-4531-9bc4-b103829b76b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:25.748339 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b666ff91-3c25-4852-8f66-b4ba11378bd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:25.750359 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b666ff91-3c25-4852-8f66-b4ba11378bd9 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:25.750359 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b666ff91-3c25-4852-8f66-b4ba11378bd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:25.750563 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b666ff91-3c25-4852-8f66-b4ba11378bd9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:25.751290 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b666ff91-3c25-4852-8f66-b4ba11378bd9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:25.754025 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b666ff91-3c25-4852-8f66-b4ba11378bd9 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:25.833499 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 864/1732] 10.222.0.102 () {60 vars in 1158 bytes} [Thu Sep 1 17:31:25 2022] POST /identity/v3/projects => generated 408 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:31:25.836051 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5771c8-fba3-4bc8-a570-0a89173c98aa None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:25.836360 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5771c8-fba3-4bc8-a570-0a89173c98aa None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:25.836674 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5771c8-fba3-4bc8-a570-0a89173c98aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:25.856678 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214e0c5c-a125-4531-9bc4-b103829b76b8 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:25.856880 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:25.880603 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-2c5771c8-fba3-4bc8-a570-0a89173c98aa None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:25.883204 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-2c5771c8-fba3-4bc8-a570-0a89173c98aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:26.002724 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70990e1-bb5e-4d3c-8c9c-c4b6e9fef2cb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f8293316baf4774b62fc11d55d50865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:26.008252 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70990e1-bb5e-4d3c-8c9c-c4b6e9fef2cb service nova] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:26.009154 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70990e1-bb5e-4d3c-8c9c-c4b6e9fef2cb service nova] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:26.009466 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70990e1-bb5e-4d3c-8c9c-c4b6e9fef2cb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:26.014627 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-e70990e1-bb5e-4d3c-8c9c-c4b6e9fef2cb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:26.018919 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 865/1733] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:25 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:26.019616 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5771c8-fba3-4bc8-a570-0a89173c98aa None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:26.027530 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-2c5771c8-fba3-4bc8-a570-0a89173c98aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:26.056962 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5771c8-fba3-4bc8-a570-0a89173c98aa None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:26.089251 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26446db0-1459-482e-80f5-7fcdb9b3d591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:26.090199 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26446db0-1459-482e-80f5-7fcdb9b3d591 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:26.090684 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26446db0-1459-482e-80f5-7fcdb9b3d591 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:26.091133 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26446db0-1459-482e-80f5-7fcdb9b3d591 admin admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:26.091945 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26446db0-1459-482e-80f5-7fcdb9b3d591 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:26.095225 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26446db0-1459-482e-80f5-7fcdb9b3d591 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:26.123539 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70990e1-bb5e-4d3c-8c9c-c4b6e9fef2cb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:26.127354 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70990e1-bb5e-4d3c-8c9c-c4b6e9fef2cb service nova] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:26.132111 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 866/1734] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:31:26 2022] POST /identity/v3/users => generated 311 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:31:26.134594 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc601c5-ead5-4304-94a9-c515431553bf None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:26.135307 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc601c5-ead5-4304-94a9-c515431553bf None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:26.137524 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc601c5-ead5-4304-94a9-c515431553bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:26.172509 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-acc601c5-ead5-4304-94a9-c515431553bf None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:26.173901 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 869/1735] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:31:25 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:31:26.174038 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-acc601c5-ead5-4304-94a9-c515431553bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:26.176959 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70990e1-bb5e-4d3c-8c9c-c4b6e9fef2cb service nova] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:26.200699 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 867/1736] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:26 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:26.202828 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70990e1-bb5e-4d3c-8c9c-c4b6e9fef2cb service nova] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:26.208490 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc601c5-ead5-4304-94a9-c515431553bf None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:26.218852 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997ed50d-970e-402c-a112-ed88ffa6da1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:26.220012 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997ed50d-970e-402c-a112-ed88ffa6da1f admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:26.220297 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997ed50d-970e-402c-a112-ed88ffa6da1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:26.222794 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997ed50d-970e-402c-a112-ed88ffa6da1f admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:26.223643 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997ed50d-970e-402c-a112-ed88ffa6da1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:26.228231 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997ed50d-970e-402c-a112-ed88ffa6da1f admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:26.231901 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc601c5-ead5-4304-94a9-c515431553bf None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:26.268589 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b8e76f-bc2a-42c5-bd24-9374fc68a82a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:26.269765 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b8e76f-bc2a-42c5-bd24-9374fc68a82a admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:26.271330 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b8e76f-bc2a-42c5-bd24-9374fc68a82a admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:26.271632 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b8e76f-bc2a-42c5-bd24-9374fc68a82a admin admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:26.273704 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b8e76f-bc2a-42c5-bd24-9374fc68a82a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:31:26.279201 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b8e76f-bc2a-42c5-bd24-9374fc68a82a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:26.279342 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 870/1737] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:31:26 2022] GET /identity/v3/roles => generated 1427 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:31:26.279428 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b8e76f-bc2a-42c5-bd24-9374fc68a82a admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:26.279504 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997ed50d-970e-402c-a112-ed88ffa6da1f admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:26.299417 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997ed50d-970e-402c-a112-ed88ffa6da1f admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:26.324971 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 868/1738] 10.222.0.102 () {58 vars in 1158 bytes} [Thu Sep 1 17:31:26 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:31:26.324971 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b8e76f-bc2a-42c5-bd24-9374fc68a82a admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:26.335374 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004fed60-35a8-4ba7-8191-c07e925e0de8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:26.335374 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004fed60-35a8-4ba7-8191-c07e925e0de8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:26.335374 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004fed60-35a8-4ba7-8191-c07e925e0de8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:26.335374 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004fed60-35a8-4ba7-8191-c07e925e0de8 admin admin] PATH_INFO: `/v3/projects/cf62ae9589e742d6b6974888e8fb175d/users/1ac8bef352c24ea5868d1815d5d5d529/roles/36577a6f46d642d4b81805ede9872490` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:26.341750 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:26.348407 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b8e76f-bc2a-42c5-bd24-9374fc68a82a admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:26.363071 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b428d449-268b-42e0-8d56-35440f579a60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:26.363722 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b428d449-268b-42e0-8d56-35440f579a60 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:26.364296 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b428d449-268b-42e0-8d56-35440f579a60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:26.364583 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b428d449-268b-42e0-8d56-35440f579a60 admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:26.365226 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b428d449-268b-42e0-8d56-35440f579a60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:26.367188 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b428d449-268b-42e0-8d56-35440f579a60 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:26.377782 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004fed60-35a8-4ba7-8191-c07e925e0de8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf62ae9589e742d6b6974888e8fb175d, user_id=1ac8bef352c24ea5868d1815d5d5d529, role_id=36577a6f46d642d4b81805ede9872490)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:26.382589 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004fed60-35a8-4ba7-8191-c07e925e0de8 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:26.437490 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 871/1739] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:31:26 2022] PUT /identity/v3/projects/cf62ae9589e742d6b6974888e8fb175d/users/1ac8bef352c24ea5868d1815d5d5d529/roles/36577a6f46d642d4b81805ede9872490 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:26.440085 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 869/1740] 10.222.0.102 () {60 vars in 1158 bytes} [Thu Sep 1 17:31:26 2022] POST /identity/v3/projects => generated 382 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:31:26.440248 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004fed60-35a8-4ba7-8191-c07e925e0de8 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:26.442510 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b428d449-268b-42e0-8d56-35440f579a60 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:26.585274 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b428d449-268b-42e0-8d56-35440f579a60 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:26.610015 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c10f326-cac4-4b79-a63d-af321f983bc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:26.615240 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c10f326-cac4-4b79-a63d-af321f983bc4 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:26.615528 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c10f326-cac4-4b79-a63d-af321f983bc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:26.615818 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c10f326-cac4-4b79-a63d-af321f983bc4 admin admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:26.616449 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c10f326-cac4-4b79-a63d-af321f983bc4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:26.620419 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c10f326-cac4-4b79-a63d-af321f983bc4 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:26.672814 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004fed60-35a8-4ba7-8191-c07e925e0de8 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:26.673904 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 870/1741] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:31:26 2022] POST /identity/v3/users => generated 292 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:31:26.678189 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c10f326-cac4-4b79-a63d-af321f983bc4 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:26.699125 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c10f326-cac4-4b79-a63d-af321f983bc4 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:26.699883 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b4f004-57f2-4b65-ac49-596f239a9dcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:26.700512 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b4f004-57f2-4b65-ac49-596f239a9dcc admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:26.700800 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b4f004-57f2-4b65-ac49-596f239a9dcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:26.701077 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b4f004-57f2-4b65-ac49-596f239a9dcc admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:26.704546 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b4f004-57f2-4b65-ac49-596f239a9dcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:26.706226 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b4f004-57f2-4b65-ac49-596f239a9dcc admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:26.710023 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86354d0b-70c6-4ca2-9889-e1a0c77468b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:26.712534 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86354d0b-70c6-4ca2-9889-e1a0c77468b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:26.713206 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86354d0b-70c6-4ca2-9889-e1a0c77468b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:26.713491 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86354d0b-70c6-4ca2-9889-e1a0c77468b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:26.714106 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86354d0b-70c6-4ca2-9889-e1a0c77468b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:26.715801 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86354d0b-70c6-4ca2-9889-e1a0c77468b0 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:26.747651 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 872/1742] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:31:26 2022] GET /identity/v3/roles => generated 1427 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:31:26.751349 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 871/1743] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:31:26 2022] GET /identity/v3/roles => generated 1427 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 17:31:26.751771 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b4f004-57f2-4b65-ac49-596f239a9dcc admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:26.752250 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86354d0b-70c6-4ca2-9889-e1a0c77468b0 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:26.772278 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b4f004-57f2-4b65-ac49-596f239a9dcc admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:26.785333 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa10044-beb2-4d07-8f49-0c873bd681c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:26.786131 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa10044-beb2-4d07-8f49-0c873bd681c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:26.786415 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa10044-beb2-4d07-8f49-0c873bd681c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:26.786686 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa10044-beb2-4d07-8f49-0c873bd681c4 admin admin] PATH_INFO: `/v3/projects/cf62ae9589e742d6b6974888e8fb175d/users/1ac8bef352c24ea5868d1815d5d5d529/roles/36577a6f46d642d4b81805ede9872490` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:26.788850 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86354d0b-70c6-4ca2-9889-e1a0c77468b0 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:26.792140 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa10044-beb2-4d07-8f49-0c873bd681c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf62ae9589e742d6b6974888e8fb175d, user_id=1ac8bef352c24ea5868d1815d5d5d529, role_id=36577a6f46d642d4b81805ede9872490)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:26.794930 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa10044-beb2-4d07-8f49-0c873bd681c4 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:26.802840 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ebe21f-7a91-4452-a9f5-8276e9c1d875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:26.803464 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ebe21f-7a91-4452-a9f5-8276e9c1d875 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:26.803748 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ebe21f-7a91-4452-a9f5-8276e9c1d875 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:26.804026 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ebe21f-7a91-4452-a9f5-8276e9c1d875 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:26.804657 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ebe21f-7a91-4452-a9f5-8276e9c1d875 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:26.810321 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ebe21f-7a91-4452-a9f5-8276e9c1d875 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:26.845030 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 873/1744] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:31:26 2022] PUT /identity/v3/projects/cf62ae9589e742d6b6974888e8fb175d/users/1ac8bef352c24ea5868d1815d5d5d529/roles/36577a6f46d642d4b81805ede9872490 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:26.845789 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 872/1745] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:31:26 2022] GET /identity/v3/roles => generated 1427 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:31:26.851871 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57555df8-f5a7-4924-b954-5aab4634659b None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:26.851871 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57555df8-f5a7-4924-b954-5aab4634659b None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:26.851871 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57555df8-f5a7-4924-b954-5aab4634659b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:26.853390 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ebe21f-7a91-4452-a9f5-8276e9c1d875 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:26.896838 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ebe21f-7a91-4452-a9f5-8276e9c1d875 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:26.923326 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:26.937408 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-57555df8-f5a7-4924-b954-5aab4634659b None None] MFA Rules not processed for user `1ac8bef352c24ea5868d1815d5d5d529`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:26.942723 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-57555df8-f5a7-4924-b954-5aab4634659b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:27.086737 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf9944c-26bd-4995-92bf-d4ebb1988d41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:27.089424 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf9944c-26bd-4995-92bf-d4ebb1988d41 admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:27.099461 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf9944c-26bd-4995-92bf-d4ebb1988d41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:27.103159 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf9944c-26bd-4995-92bf-d4ebb1988d41 admin admin] PATH_INFO: `/v3/projects/dc2a24c4510c4e958fd29431931bc272/users/819117761b2b4969b260f3d30f2273b2/roles/b75cb360925e4ba5a65f14965c103ffd` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:27.124587 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf9944c-26bd-4995-92bf-d4ebb1988d41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc2a24c4510c4e958fd29431931bc272, user_id=819117761b2b4969b260f3d30f2273b2, role_id=b75cb360925e4ba5a65f14965c103ffd)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:27.127546 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf9944c-26bd-4995-92bf-d4ebb1988d41 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:27.154274 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 873/1746] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:31:26 2022] PUT /identity/v3/projects/dc2a24c4510c4e958fd29431931bc272/users/819117761b2b4969b260f3d30f2273b2/roles/b75cb360925e4ba5a65f14965c103ffd => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:27.165114 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e66477-3fdf-499b-972b-62ef3bf97643 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:27.165468 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e66477-3fdf-499b-972b-62ef3bf97643 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:27.165587 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e66477-3fdf-499b-972b-62ef3bf97643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:27.239337 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-49e66477-3fdf-499b-972b-62ef3bf97643 None None] MFA Rules not processed for user `819117761b2b4969b260f3d30f2273b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:27.246140 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-49e66477-3fdf-499b-972b-62ef3bf97643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:27.255292 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 874/1747] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:26 2022] POST /identity/v3/auth/tokens => generated 3361 bytes in 407 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:27.269515 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57555df8-f5a7-4924-b954-5aab4634659b None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:27.273543 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-57555df8-f5a7-4924-b954-5aab4634659b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:27.440700 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57555df8-f5a7-4924-b954-5aab4634659b None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:27.443554 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 874/1748] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:27 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:31:27.449228 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e66477-3fdf-499b-972b-62ef3bf97643 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:27.457658 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:27.582246 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7545ac-9852-4bbf-bc22-7e764d12cea4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '523ed589872a45a09b6a7675a7fb0b75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:27.582984 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7545ac-9852-4bbf-bc22-7e764d12cea4 service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:27.583371 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7545ac-9852-4bbf-bc22-7e764d12cea4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:27.583799 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7545ac-9852-4bbf-bc22-7e764d12cea4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:27.607650 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e66477-3fdf-499b-972b-62ef3bf97643 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:27.622178 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be78cb7-816b-4a4d-9550-b4aaa0af4329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:27.622930 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be78cb7-816b-4a4d-9550-b4aaa0af4329 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:27.623360 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be78cb7-816b-4a4d-9550-b4aaa0af4329 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:27.623786 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be78cb7-816b-4a4d-9550-b4aaa0af4329 admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:27.624602 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be78cb7-816b-4a4d-9550-b4aaa0af4329 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:27.626762 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be78cb7-816b-4a4d-9550-b4aaa0af4329 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:27.696399 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 875/1749] 10.222.0.102 () {60 vars in 1158 bytes} [Thu Sep 1 17:31:27 2022] POST /identity/v3/projects => generated 382 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:31:27.697402 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be78cb7-816b-4a4d-9550-b4aaa0af4329 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:27.720038 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7545ac-9852-4bbf-bc22-7e764d12cea4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:27.722693 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7545ac-9852-4bbf-bc22-7e764d12cea4 service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:27.737782 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 875/1750] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:31:27 2022] GET /identity/v3/auth/tokens => generated 3361 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:31:27.739982 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:27.926017 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be78cb7-816b-4a4d-9550-b4aaa0af4329 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:27.936116 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6333631e-c34a-4480-a4c2-059ab31a1149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:27.939342 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6333631e-c34a-4480-a4c2-059ab31a1149 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:27.939342 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6333631e-c34a-4480-a4c2-059ab31a1149 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:27.939342 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6333631e-c34a-4480-a4c2-059ab31a1149 admin admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:27.939972 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6333631e-c34a-4480-a4c2-059ab31a1149 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:27.946015 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6333631e-c34a-4480-a4c2-059ab31a1149 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:28.005040 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7545ac-9852-4bbf-bc22-7e764d12cea4 service cinder] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:28.016796 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 876/1751] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:31:27 2022] POST /identity/v3/users => generated 298 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:31:28.032281 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7545ac-9852-4bbf-bc22-7e764d12cea4 service cinder] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:28.046069 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a8ef05-8a70-4074-a257-42945f0087a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:28.047275 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a8ef05-8a70-4074-a257-42945f0087a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:28.048504 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a8ef05-8a70-4074-a257-42945f0087a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:28.048797 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a8ef05-8a70-4074-a257-42945f0087a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:28.058293 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a8ef05-8a70-4074-a257-42945f0087a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:28.065424 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a8ef05-8a70-4074-a257-42945f0087a4 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:28.103291 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6333631e-c34a-4480-a4c2-059ab31a1149 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:28.105965 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-6333631e-c34a-4480-a4c2-059ab31a1149 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:28.115978 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 876/1752] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:31:28 2022] GET /identity/v3/roles => generated 1427 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:31:28.119975 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a8ef05-8a70-4074-a257-42945f0087a4 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:28.167703 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a8ef05-8a70-4074-a257-42945f0087a4 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:28.194667 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182beb76-2b03-4a34-988d-0e4e350312ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:28.206609 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182beb76-2b03-4a34-988d-0e4e350312ef admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:28.208953 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182beb76-2b03-4a34-988d-0e4e350312ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:28.211015 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182beb76-2b03-4a34-988d-0e4e350312ef admin admin] PATH_INFO: `/v3/projects/546c57ccc877473c99a212984c372f51/users/f1bf536db604446e9269b6acf5db57ba/roles/36577a6f46d642d4b81805ede9872490` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:28.275140 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182beb76-2b03-4a34-988d-0e4e350312ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=546c57ccc877473c99a212984c372f51, user_id=f1bf536db604446e9269b6acf5db57ba, role_id=36577a6f46d642d4b81805ede9872490)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:28.281070 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182beb76-2b03-4a34-988d-0e4e350312ef admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:28.298948 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 877/1753] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:31:28 2022] PUT /identity/v3/projects/546c57ccc877473c99a212984c372f51/users/f1bf536db604446e9269b6acf5db57ba/roles/36577a6f46d642d4b81805ede9872490 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:28.303128 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182beb76-2b03-4a34-988d-0e4e350312ef admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:28.343917 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6333631e-c34a-4480-a4c2-059ab31a1149 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:28.457901 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4e714a-f06b-4062-8ecc-2f7472ed9cbd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '523ed589872a45a09b6a7675a7fb0b75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:28.458685 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4e714a-f06b-4062-8ecc-2f7472ed9cbd service cinder] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:28.458977 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4e714a-f06b-4062-8ecc-2f7472ed9cbd service cinder] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:28.459758 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4e714a-f06b-4062-8ecc-2f7472ed9cbd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:28.462049 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-2c4e714a-f06b-4062-8ecc-2f7472ed9cbd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:28.539700 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182beb76-2b03-4a34-988d-0e4e350312ef admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:28.571677 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6085f662-8cdd-44e1-9bca-d6c5cccaa48d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:28.573736 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6085f662-8cdd-44e1-9bca-d6c5cccaa48d admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:28.575226 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6085f662-8cdd-44e1-9bca-d6c5cccaa48d admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:28.576146 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6085f662-8cdd-44e1-9bca-d6c5cccaa48d admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:28.579099 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6085f662-8cdd-44e1-9bca-d6c5cccaa48d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:28.581252 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6085f662-8cdd-44e1-9bca-d6c5cccaa48d admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:28.588006 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4e714a-f06b-4062-8ecc-2f7472ed9cbd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:28.589652 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4e714a-f06b-4062-8ecc-2f7472ed9cbd service cinder] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:28.598057 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 878/1754] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:31:28 2022] GET /identity/v3/roles => generated 1427 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 17:31:28.605594 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6085f662-8cdd-44e1-9bca-d6c5cccaa48d admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:28.635115 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6085f662-8cdd-44e1-9bca-d6c5cccaa48d admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:28.642203 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b73b79e-941b-4f94-9182-2f73e6514a60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:28.647001 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b73b79e-941b-4f94-9182-2f73e6514a60 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:28.647406 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b73b79e-941b-4f94-9182-2f73e6514a60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:28.647934 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 877/1755] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:31:28 2022] GET /identity/v3/auth/tokens => generated 3294 bytes in 545 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 17:31:28.648412 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b73b79e-941b-4f94-9182-2f73e6514a60 admin admin] PATH_INFO: `/v3/projects/546c57ccc877473c99a212984c372f51/users/f1bf536db604446e9269b6acf5db57ba/roles/36577a6f46d642d4b81805ede9872490` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:28.660084 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b73b79e-941b-4f94-9182-2f73e6514a60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=546c57ccc877473c99a212984c372f51, user_id=f1bf536db604446e9269b6acf5db57ba, role_id=36577a6f46d642d4b81805ede9872490)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:28.665838 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b73b79e-941b-4f94-9182-2f73e6514a60 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:28.712523 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 879/1756] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:31:28 2022] PUT /identity/v3/projects/546c57ccc877473c99a212984c372f51/users/f1bf536db604446e9269b6acf5db57ba/roles/36577a6f46d642d4b81805ede9872490 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:28.725566 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55a89ed-38b2-4440-9325-1acfd028face None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:28.726035 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55a89ed-38b2-4440-9325-1acfd028face None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:28.726364 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55a89ed-38b2-4440-9325-1acfd028face None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:28.791249 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:28.827740 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-b55a89ed-38b2-4440-9325-1acfd028face None None] MFA Rules not processed for user `f1bf536db604446e9269b6acf5db57ba`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:28.830231 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-b55a89ed-38b2-4440-9325-1acfd028face None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:28.998170 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 878/1757] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:28 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:29.110442 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b73b79e-941b-4f94-9182-2f73e6514a60 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:29.116317 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:29.310152 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b73b79e-941b-4f94-9182-2f73e6514a60 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:29.322440 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e43ec36-fa8e-4328-b78a-fed784b731fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '523ed589872a45a09b6a7675a7fb0b75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:29.323514 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e43ec36-fa8e-4328-b78a-fed784b731fa service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:29.323514 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e43ec36-fa8e-4328-b78a-fed784b731fa service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:29.323707 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e43ec36-fa8e-4328-b78a-fed784b731fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:29.363860 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e43ec36-fa8e-4328-b78a-fed784b731fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:29.366680 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e43ec36-fa8e-4328-b78a-fed784b731fa service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:29.397629 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 880/1758] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:31:29 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:31:29.976061 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55a89ed-38b2-4440-9325-1acfd028face None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:29.994591 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55a89ed-38b2-4440-9325-1acfd028face None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:30.006814 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d24ef4f-50b1-4d30-bc8e-ba658f2c50a6 tempest-VolumeQuotaClassesTest-1729218546 tempest-VolumeQuotaClassesTest-1729218546-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1bf536db604446e9269b6acf5db57ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546c57ccc877473c99a212984c372f51', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:30.007894 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d24ef4f-50b1-4d30-bc8e-ba658f2c50a6 tempest-VolumeQuotaClassesTest-1729218546 tempest-VolumeQuotaClassesTest-1729218546-project-admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:30.008670 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d24ef4f-50b1-4d30-bc8e-ba658f2c50a6 tempest-VolumeQuotaClassesTest-1729218546 tempest-VolumeQuotaClassesTest-1729218546-project-admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:30.008966 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d24ef4f-50b1-4d30-bc8e-ba658f2c50a6 tempest-VolumeQuotaClassesTest-1729218546 tempest-VolumeQuotaClassesTest-1729218546-project-admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:30.009888 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d24ef4f-50b1-4d30-bc8e-ba658f2c50a6 tempest-VolumeQuotaClassesTest-1729218546 tempest-VolumeQuotaClassesTest-1729218546-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:31:30.010204 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d24ef4f-50b1-4d30-bc8e-ba658f2c50a6 tempest-VolumeQuotaClassesTest-1729218546 tempest-VolumeQuotaClassesTest-1729218546-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:30.011816 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d24ef4f-50b1-4d30-bc8e-ba658f2c50a6 tempest-VolumeQuotaClassesTest-1729218546 tempest-VolumeQuotaClassesTest-1729218546-project-admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:30.035167 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 879/1759] 10.222.0.102 () {58 vars in 1158 bytes} [Thu Sep 1 17:31:29 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:31:30.038604 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e43ec36-fa8e-4328-b78a-fed784b731fa service cinder] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:30.048886 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e43ec36-fa8e-4328-b78a-fed784b731fa service cinder] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:30.054652 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a2b237-e681-430b-9a98-325f9e4d459c tempest-VolumeQuotaClassesTest-1729218546 tempest-VolumeQuotaClassesTest-1729218546-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1bf536db604446e9269b6acf5db57ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546c57ccc877473c99a212984c372f51', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:30.055399 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a2b237-e681-430b-9a98-325f9e4d459c tempest-VolumeQuotaClassesTest-1729218546 tempest-VolumeQuotaClassesTest-1729218546-project-admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:30.055749 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a2b237-e681-430b-9a98-325f9e4d459c tempest-VolumeQuotaClassesTest-1729218546 tempest-VolumeQuotaClassesTest-1729218546-project-admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:30.056196 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a2b237-e681-430b-9a98-325f9e4d459c tempest-VolumeQuotaClassesTest-1729218546 tempest-VolumeQuotaClassesTest-1729218546-project-admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:30.056840 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a2b237-e681-430b-9a98-325f9e4d459c tempest-VolumeQuotaClassesTest-1729218546 tempest-VolumeQuotaClassesTest-1729218546-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:30.058874 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a2b237-e681-430b-9a98-325f9e4d459c tempest-VolumeQuotaClassesTest-1729218546 tempest-VolumeQuotaClassesTest-1729218546-project-admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:30.131065 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 881/1760] 10.222.0.102 () {60 vars in 1158 bytes} [Thu Sep 1 17:31:30 2022] POST /identity/v3/projects => generated 355 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:31:30.135061 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d24ef4f-50b1-4d30-bc8e-ba658f2c50a6 tempest-VolumeQuotaClassesTest-1729218546 tempest-VolumeQuotaClassesTest-1729218546-project-admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:30.149182 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d24ef4f-50b1-4d30-bc8e-ba658f2c50a6 tempest-VolumeQuotaClassesTest-1729218546 tempest-VolumeQuotaClassesTest-1729218546-project-admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:30.236087 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acad7823-cfd7-4fcd-a934-9fed93528ca3 tempest-VolumeQuotaClassesTest-1729218546 tempest-VolumeQuotaClassesTest-1729218546-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1bf536db604446e9269b6acf5db57ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546c57ccc877473c99a212984c372f51', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:30.252078 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acad7823-cfd7-4fcd-a934-9fed93528ca3 tempest-VolumeQuotaClassesTest-1729218546 tempest-VolumeQuotaClassesTest-1729218546-project-admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:30.252355 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acad7823-cfd7-4fcd-a934-9fed93528ca3 tempest-VolumeQuotaClassesTest-1729218546 tempest-VolumeQuotaClassesTest-1729218546-project-admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:30.252628 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acad7823-cfd7-4fcd-a934-9fed93528ca3 tempest-VolumeQuotaClassesTest-1729218546 tempest-VolumeQuotaClassesTest-1729218546-project-admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:30.253293 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acad7823-cfd7-4fcd-a934-9fed93528ca3 tempest-VolumeQuotaClassesTest-1729218546 tempest-VolumeQuotaClassesTest-1729218546-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:31:30.253637 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acad7823-cfd7-4fcd-a934-9fed93528ca3 tempest-VolumeQuotaClassesTest-1729218546 tempest-VolumeQuotaClassesTest-1729218546-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:30.255373 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acad7823-cfd7-4fcd-a934-9fed93528ca3 tempest-VolumeQuotaClassesTest-1729218546 tempest-VolumeQuotaClassesTest-1729218546-project-admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:30.273428 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 880/1761] 10.222.0.102 () {58 vars in 1158 bytes} [Thu Sep 1 17:31:30 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:31:30.316935 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a2b237-e681-430b-9a98-325f9e4d459c tempest-VolumeQuotaClassesTest-1729218546 tempest-VolumeQuotaClassesTest-1729218546-project-admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:30.331136 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a2b237-e681-430b-9a98-325f9e4d459c tempest-VolumeQuotaClassesTest-1729218546 tempest-VolumeQuotaClassesTest-1729218546-project-admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:30.362780 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505a63be-fe27-4454-ad98-0ceca65c78fa tempest-VolumeQuotaClassesTest-1729218546 tempest-VolumeQuotaClassesTest-1729218546-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1bf536db604446e9269b6acf5db57ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546c57ccc877473c99a212984c372f51', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:30.368612 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505a63be-fe27-4454-ad98-0ceca65c78fa tempest-VolumeQuotaClassesTest-1729218546 tempest-VolumeQuotaClassesTest-1729218546-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:30.370068 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505a63be-fe27-4454-ad98-0ceca65c78fa tempest-VolumeQuotaClassesTest-1729218546 tempest-VolumeQuotaClassesTest-1729218546-project-admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:30.370510 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505a63be-fe27-4454-ad98-0ceca65c78fa tempest-VolumeQuotaClassesTest-1729218546 tempest-VolumeQuotaClassesTest-1729218546-project-admin] PATH_INFO: `/v3/projects/246eedcdfa194b759d9a9350e2fff49b` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:30.374792 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505a63be-fe27-4454-ad98-0ceca65c78fa tempest-VolumeQuotaClassesTest-1729218546 tempest-VolumeQuotaClassesTest-1729218546-project-admin] RBAC: Authorizing `identity:delete_project(project_id=246eedcdfa194b759d9a9350e2fff49b)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:30.391332 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505a63be-fe27-4454-ad98-0ceca65c78fa tempest-VolumeQuotaClassesTest-1729218546 tempest-VolumeQuotaClassesTest-1729218546-project-admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:30.586236 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-505a63be-fe27-4454-ad98-0ceca65c78fa tempest-VolumeQuotaClassesTest-1729218546 tempest-VolumeQuotaClassesTest-1729218546-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '246eedcdfa194b759d9a9350e2fff49b', 'request_id': 'req-505a63be-fe27-4454-ad98-0ceca65c78fa'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:30.604271 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-505a63be-fe27-4454-ad98-0ceca65c78fa tempest-VolumeQuotaClassesTest-1729218546 tempest-VolumeQuotaClassesTest-1729218546-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '246eedcdfa194b759d9a9350e2fff49b', 'request_id': 'req-505a63be-fe27-4454-ad98-0ceca65c78fa'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:30.612731 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-505a63be-fe27-4454-ad98-0ceca65c78fa tempest-VolumeQuotaClassesTest-1729218546 tempest-VolumeQuotaClassesTest-1729218546-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '246eedcdfa194b759d9a9350e2fff49b', 'request_id': 'req-505a63be-fe27-4454-ad98-0ceca65c78fa'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:30.637291 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-505a63be-fe27-4454-ad98-0ceca65c78fa tempest-VolumeQuotaClassesTest-1729218546 tempest-VolumeQuotaClassesTest-1729218546-project-admin] The token cache is being invalidate because project 246eedcdfa194b759d9a9350e2fff49b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:31:30.637659 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-505a63be-fe27-4454-ad98-0ceca65c78fa tempest-VolumeQuotaClassesTest-1729218546 tempest-VolumeQuotaClassesTest-1729218546-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:30.639777 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-505a63be-fe27-4454-ad98-0ceca65c78fa tempest-VolumeQuotaClassesTest-1729218546 tempest-VolumeQuotaClassesTest-1729218546-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:30.649262 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 882/1762] 10.222.0.102 () {58 vars in 1238 bytes} [Thu Sep 1 17:31:30 2022] DELETE /identity/v3/projects/246eedcdfa194b759d9a9350e2fff49b => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:31.057716 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acad7823-cfd7-4fcd-a934-9fed93528ca3 tempest-VolumeQuotaClassesTest-1729218546 tempest-VolumeQuotaClassesTest-1729218546-project-admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:31.059742 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-acad7823-cfd7-4fcd-a934-9fed93528ca3 tempest-VolumeQuotaClassesTest-1729218546 tempest-VolumeQuotaClassesTest-1729218546-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:31.128776 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acad7823-cfd7-4fcd-a934-9fed93528ca3 tempest-VolumeQuotaClassesTest-1729218546 tempest-VolumeQuotaClassesTest-1729218546-project-admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:31.171287 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:31.225468 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628214b4-1b43-415e-b13d-f1f6eaa49e45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:31.226109 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628214b4-1b43-415e-b13d-f1f6eaa49e45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:31.226384 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628214b4-1b43-415e-b13d-f1f6eaa49e45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:31.226679 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628214b4-1b43-415e-b13d-f1f6eaa49e45 admin admin] PATH_INFO: `/v3/users/819117761b2b4969b260f3d30f2273b2` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:31.227875 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628214b4-1b43-415e-b13d-f1f6eaa49e45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=819117761b2b4969b260f3d30f2273b2)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:31.230032 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628214b4-1b43-415e-b13d-f1f6eaa49e45 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:31.293937 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-628214b4-1b43-415e-b13d-f1f6eaa49e45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '819117761b2b4969b260f3d30f2273b2'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:31.294632 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-628214b4-1b43-415e-b13d-f1f6eaa49e45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '819117761b2b4969b260f3d30f2273b2'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:31.311839 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-628214b4-1b43-415e-b13d-f1f6eaa49e45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '819117761b2b4969b260f3d30f2273b2'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:31.325945 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-628214b4-1b43-415e-b13d-f1f6eaa49e45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '819117761b2b4969b260f3d30f2273b2'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:31.327605 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-628214b4-1b43-415e-b13d-f1f6eaa49e45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '819117761b2b4969b260f3d30f2273b2'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:31.343386 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 881/1763] 10.222.0.102 () {58 vars in 1229 bytes} [Thu Sep 1 17:31:31 2022] DELETE /identity/v3/users/819117761b2b4969b260f3d30f2273b2 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:31.347830 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe3c7e8-9742-4f34-8ecf-08ad7fc04466 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:31.348739 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe3c7e8-9742-4f34-8ecf-08ad7fc04466 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:31.349521 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe3c7e8-9742-4f34-8ecf-08ad7fc04466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:31.374361 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-cbe3c7e8-9742-4f34-8ecf-08ad7fc04466 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:31.375789 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-cbe3c7e8-9742-4f34-8ecf-08ad7fc04466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:31.510029 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 883/1764] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:31 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:31.534135 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628214b4-1b43-415e-b13d-f1f6eaa49e45 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:31.537354 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-628214b4-1b43-415e-b13d-f1f6eaa49e45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:31.684342 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628214b4-1b43-415e-b13d-f1f6eaa49e45 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:31.702196 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c749791-41c1-42d2-8bba-8f30d2ae4f8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:31.703472 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c749791-41c1-42d2-8bba-8f30d2ae4f8e service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:31.704425 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c749791-41c1-42d2-8bba-8f30d2ae4f8e service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:31.704963 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c749791-41c1-42d2-8bba-8f30d2ae4f8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:31.731289 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c749791-41c1-42d2-8bba-8f30d2ae4f8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:31.733163 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c749791-41c1-42d2-8bba-8f30d2ae4f8e service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:31.745894 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 882/1765] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:31:31 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 17:31:31.801273 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef68f1c-8385-48d0-b40c-025f5d428eb2 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:31.801455 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef68f1c-8385-48d0-b40c-025f5d428eb2 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:31.801733 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef68f1c-8385-48d0-b40c-025f5d428eb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:31.883637 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:32.007062 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-3ef68f1c-8385-48d0-b40c-025f5d428eb2 None None] MFA Rules not processed for user `575a76b99ea945efb1024fbb3dc81aaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:32.010481 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514a3266-bb02-4f49-88ca-7e7de1bad24f None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:32.011204 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514a3266-bb02-4f49-88ca-7e7de1bad24f None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:32.011833 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514a3266-bb02-4f49-88ca-7e7de1bad24f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:32.012366 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-3ef68f1c-8385-48d0-b40c-025f5d428eb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:32.071739 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-514a3266-bb02-4f49-88ca-7e7de1bad24f None None] MFA Rules not processed for user `7c7d3594fdab4191be1d841af43a5c79`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:32.078428 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-514a3266-bb02-4f49-88ca-7e7de1bad24f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:32.103979 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 883/1766] 10.222.0.73 () {58 vars in 1019 bytes} [Thu Sep 1 17:31:32 2022] POST /identity/v3/auth/tokens => generated 3131 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:31:32.283425 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514a3266-bb02-4f49-88ca-7e7de1bad24f None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:32.284898 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-514a3266-bb02-4f49-88ca-7e7de1bad24f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:32.302612 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514a3266-bb02-4f49-88ca-7e7de1bad24f None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:32.308723 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 884/1767] 10.222.0.102 () {58 vars in 1014 bytes} [Thu Sep 1 17:31:31 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 509 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:32.319651 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30779e86-caea-45c2-bd9a-98395f869964 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:32.320295 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30779e86-caea-45c2-bd9a-98395f869964 service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:32.320575 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30779e86-caea-45c2-bd9a-98395f869964 service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:32.321005 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30779e86-caea-45c2-bd9a-98395f869964 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:32.340275 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30779e86-caea-45c2-bd9a-98395f869964 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:32.342094 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30779e86-caea-45c2-bd9a-98395f869964 service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:32.359384 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 884/1768] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:31:32 2022] GET /identity/v3/auth/tokens => generated 3131 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 17:31:32.535175 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef68f1c-8385-48d0-b40c-025f5d428eb2 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:32.536670 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-3ef68f1c-8385-48d0-b40c-025f5d428eb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:32.750880 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef68f1c-8385-48d0-b40c-025f5d428eb2 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:32.766647 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f293a6a-42ae-4485-88d8-798a415ac6bd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd55edf6cd1804006aa8269b4c6f80aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:32.767597 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f293a6a-42ae-4485-88d8-798a415ac6bd service swift] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:32.768168 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f293a6a-42ae-4485-88d8-798a415ac6bd service swift] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:32.768644 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f293a6a-42ae-4485-88d8-798a415ac6bd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:32.778607 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f293a6a-42ae-4485-88d8-798a415ac6bd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:32.780629 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f293a6a-42ae-4485-88d8-798a415ac6bd service swift] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:32.814764 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 885/1769] 10.222.0.102 () {60 vars in 1389 bytes} [Thu Sep 1 17:31:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 17:31:32.916897 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30779e86-caea-45c2-bd9a-98395f869964 service neutron] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:32.918632 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-30779e86-caea-45c2-bd9a-98395f869964 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:32.935127 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30779e86-caea-45c2-bd9a-98395f869964 service neutron] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:32.940848 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8c0791-2a27-4237-978e-008a4c2ee9ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:32.941531 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8c0791-2a27-4237-978e-008a4c2ee9ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:32.941814 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8c0791-2a27-4237-978e-008a4c2ee9ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:32.942092 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8c0791-2a27-4237-978e-008a4c2ee9ab admin admin] PATH_INFO: `/v3/users/a54660f778c24611ae52c1628d17fd78` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:32.943267 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8c0791-2a27-4237-978e-008a4c2ee9ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=a54660f778c24611ae52c1628d17fd78)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:32.945265 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8c0791-2a27-4237-978e-008a4c2ee9ab admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:32.971283 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:33.027168 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-7f8c0791-2a27-4237-978e-008a4c2ee9ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a54660f778c24611ae52c1628d17fd78'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:33.034897 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-7f8c0791-2a27-4237-978e-008a4c2ee9ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a54660f778c24611ae52c1628d17fd78'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:33.064686 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-7f8c0791-2a27-4237-978e-008a4c2ee9ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a54660f778c24611ae52c1628d17fd78'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:33.087794 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-7f8c0791-2a27-4237-978e-008a4c2ee9ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a54660f778c24611ae52c1628d17fd78'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:33.089098 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-7f8c0791-2a27-4237-978e-008a4c2ee9ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a54660f778c24611ae52c1628d17fd78'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:33.134654 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 885/1770] 10.222.0.102 () {58 vars in 1229 bytes} [Thu Sep 1 17:31:32 2022] DELETE /identity/v3/users/a54660f778c24611ae52c1628d17fd78 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:33.139350 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56f89ca-7f5b-485a-9dd3-7a6d9e0ea90a None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:33.141738 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56f89ca-7f5b-485a-9dd3-7a6d9e0ea90a None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:33.142211 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56f89ca-7f5b-485a-9dd3-7a6d9e0ea90a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:33.171586 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-a56f89ca-7f5b-485a-9dd3-7a6d9e0ea90a None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:33.173099 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-a56f89ca-7f5b-485a-9dd3-7a6d9e0ea90a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:33.186093 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8c0791-2a27-4237-978e-008a4c2ee9ab admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:33.187893 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-7f8c0791-2a27-4237-978e-008a4c2ee9ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:33.357385 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:33.393036 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 886/1771] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:33 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:33.415629 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8c0791-2a27-4237-978e-008a4c2ee9ab admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:33.433938 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56f89ca-7f5b-485a-9dd3-7a6d9e0ea90a None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:33.435256 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-a56f89ca-7f5b-485a-9dd3-7a6d9e0ea90a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:33.447244 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7929c0d-0e4f-424c-9aab-7593fe0b8251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:33.447905 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7929c0d-0e4f-424c-9aab-7593fe0b8251 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:33.448187 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7929c0d-0e4f-424c-9aab-7593fe0b8251 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:33.448461 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7929c0d-0e4f-424c-9aab-7593fe0b8251 admin admin] PATH_INFO: `/v3/users/8518291fd07847aca6e86828a43deb01` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:33.449706 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7929c0d-0e4f-424c-9aab-7593fe0b8251 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8518291fd07847aca6e86828a43deb01)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:33.452480 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7929c0d-0e4f-424c-9aab-7593fe0b8251 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:33.554405 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56f89ca-7f5b-485a-9dd3-7a6d9e0ea90a None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:33.555750 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-a7929c0d-0e4f-424c-9aab-7593fe0b8251 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8518291fd07847aca6e86828a43deb01'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:33.557298 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-a7929c0d-0e4f-424c-9aab-7593fe0b8251 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8518291fd07847aca6e86828a43deb01'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:33.561818 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1414decd-b3ce-490d-8179-79a649c78a10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:33.562549 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1414decd-b3ce-490d-8179-79a649c78a10 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:33.562826 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1414decd-b3ce-490d-8179-79a649c78a10 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:33.563161 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1414decd-b3ce-490d-8179-79a649c78a10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:33.583721 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-a7929c0d-0e4f-424c-9aab-7593fe0b8251 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8518291fd07847aca6e86828a43deb01'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:33.585862 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1414decd-b3ce-490d-8179-79a649c78a10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:33.587707 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1414decd-b3ce-490d-8179-79a649c78a10 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:33.604371 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-a7929c0d-0e4f-424c-9aab-7593fe0b8251 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8518291fd07847aca6e86828a43deb01'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:33.605405 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-a7929c0d-0e4f-424c-9aab-7593fe0b8251 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8518291fd07847aca6e86828a43deb01'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:33.622045 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 887/1772] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:31:33 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:31:33.623895 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1414decd-b3ce-490d-8179-79a649c78a10 service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:33.627232 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-1414decd-b3ce-490d-8179-79a649c78a10 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:33.635494 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 886/1773] 10.222.0.102 () {58 vars in 1229 bytes} [Thu Sep 1 17:31:33 2022] DELETE /identity/v3/users/8518291fd07847aca6e86828a43deb01 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:33.640552 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540297cf-6533-44aa-9478-918b89889699 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:33.640839 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540297cf-6533-44aa-9478-918b89889699 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:33.641135 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540297cf-6533-44aa-9478-918b89889699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:33.657845 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1414decd-b3ce-490d-8179-79a649c78a10 service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:33.683507 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-540297cf-6533-44aa-9478-918b89889699 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:33.684770 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-540297cf-6533-44aa-9478-918b89889699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:33.715181 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:DELETE=18,keystone:SELECT=80,keystone:INSERT=2 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:33.788135 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e416b04c-c3c2-4845-acae-9d28e70700b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:33.791630 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e416b04c-c3c2-4845-acae-9d28e70700b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:33.791987 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e416b04c-c3c2-4845-acae-9d28e70700b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:33.792272 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e416b04c-c3c2-4845-acae-9d28e70700b8 admin admin] PATH_INFO: `/v3/projects/dc2a24c4510c4e958fd29431931bc272` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:33.795861 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e416b04c-c3c2-4845-acae-9d28e70700b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc2a24c4510c4e958fd29431931bc272)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:33.801041 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e416b04c-c3c2-4845-acae-9d28e70700b8 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:33.846348 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 887/1774] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:33 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:33.887846 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540297cf-6533-44aa-9478-918b89889699 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:33.890219 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-540297cf-6533-44aa-9478-918b89889699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:33.917871 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-e416b04c-c3c2-4845-acae-9d28e70700b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc2a24c4510c4e958fd29431931bc272', 'request_id': 'req-e416b04c-c3c2-4845-acae-9d28e70700b8'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:33.928665 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-e416b04c-c3c2-4845-acae-9d28e70700b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc2a24c4510c4e958fd29431931bc272', 'request_id': 'req-e416b04c-c3c2-4845-acae-9d28e70700b8'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:33.939261 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-e416b04c-c3c2-4845-acae-9d28e70700b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc2a24c4510c4e958fd29431931bc272', 'request_id': 'req-e416b04c-c3c2-4845-acae-9d28e70700b8'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:34.004293 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-e416b04c-c3c2-4845-acae-9d28e70700b8 admin admin] The token cache is being invalidate because project dc2a24c4510c4e958fd29431931bc272 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:31:34.004631 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-e416b04c-c3c2-4845-acae-9d28e70700b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:34.007328 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-e416b04c-c3c2-4845-acae-9d28e70700b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:34.015048 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540297cf-6533-44aa-9478-918b89889699 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:34.020227 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 888/1775] 10.222.0.102 () {58 vars in 1238 bytes} [Thu Sep 1 17:31:33 2022] DELETE /identity/v3/projects/dc2a24c4510c4e958fd29431931bc272 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:34.026825 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e416b04c-c3c2-4845-acae-9d28e70700b8 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:34.028664 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-e416b04c-c3c2-4845-acae-9d28e70700b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:34.079820 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:34.159631 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83137439-5c05-4144-831b-0057b325e5d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:34.160598 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83137439-5c05-4144-831b-0057b325e5d5 service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:34.160977 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83137439-5c05-4144-831b-0057b325e5d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:34.161294 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83137439-5c05-4144-831b-0057b325e5d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:34.175973 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-83137439-5c05-4144-831b-0057b325e5d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:34.211552 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83137439-5c05-4144-831b-0057b325e5d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:34.211754 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e416b04c-c3c2-4845-acae-9d28e70700b8 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:34.211894 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83137439-5c05-4144-831b-0057b325e5d5 service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:34.223096 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf4f7d1-f199-49f9-ad89-329c124c03d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:34.224589 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf4f7d1-f199-49f9-ad89-329c124c03d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:34.226901 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf4f7d1-f199-49f9-ad89-329c124c03d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:34.227905 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf4f7d1-f199-49f9-ad89-329c124c03d2 admin admin] PATH_INFO: `/v3/users/f1bf536db604446e9269b6acf5db57ba` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:34.232171 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf4f7d1-f199-49f9-ad89-329c124c03d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1bf536db604446e9269b6acf5db57ba)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:34.245930 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf4f7d1-f199-49f9-ad89-329c124c03d2 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:34.246470 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 888/1776] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:31:33 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:31:34.391565 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-cdf4f7d1-f199-49f9-ad89-329c124c03d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1bf536db604446e9269b6acf5db57ba'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:34.435632 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-cdf4f7d1-f199-49f9-ad89-329c124c03d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1bf536db604446e9269b6acf5db57ba'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:34.451872 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-cdf4f7d1-f199-49f9-ad89-329c124c03d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1bf536db604446e9269b6acf5db57ba'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:34.453532 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-cdf4f7d1-f199-49f9-ad89-329c124c03d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1bf536db604446e9269b6acf5db57ba'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:34.454322 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-cdf4f7d1-f199-49f9-ad89-329c124c03d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1bf536db604446e9269b6acf5db57ba'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:34.510861 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 889/1777] 10.222.0.102 () {58 vars in 1229 bytes} [Thu Sep 1 17:31:34 2022] DELETE /identity/v3/users/f1bf536db604446e9269b6acf5db57ba => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:35.349203 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83137439-5c05-4144-831b-0057b325e5d5 service neutron] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:35.352521 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-83137439-5c05-4144-831b-0057b325e5d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:35.368972 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83137439-5c05-4144-831b-0057b325e5d5 service neutron] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:35.390536 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf4f7d1-f199-49f9-ad89-329c124c03d2 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:35.393552 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-cdf4f7d1-f199-49f9-ad89-329c124c03d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:35.435923 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edc2356-be91-4ba7-ac40-723b20961a1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:35.436570 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edc2356-be91-4ba7-ac40-723b20961a1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:35.436884 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edc2356-be91-4ba7-ac40-723b20961a1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:35.437122 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edc2356-be91-4ba7-ac40-723b20961a1f admin admin] PATH_INFO: `/v3/projects/94689412ae164432a18b7444b74c2bae` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:35.438389 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edc2356-be91-4ba7-ac40-723b20961a1f admin admin] RBAC: Authorizing `identity:delete_project(project_id=94689412ae164432a18b7444b74c2bae)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:35.440249 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edc2356-be91-4ba7-ac40-723b20961a1f admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:35.462401 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:35.502138 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf4f7d1-f199-49f9-ad89-329c124c03d2 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:35.517814 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-3edc2356-be91-4ba7-ac40-723b20961a1f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94689412ae164432a18b7444b74c2bae', 'request_id': 'req-3edc2356-be91-4ba7-ac40-723b20961a1f'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:35.532528 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-3edc2356-be91-4ba7-ac40-723b20961a1f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94689412ae164432a18b7444b74c2bae', 'request_id': 'req-3edc2356-be91-4ba7-ac40-723b20961a1f'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:35.551476 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-3edc2356-be91-4ba7-ac40-723b20961a1f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94689412ae164432a18b7444b74c2bae', 'request_id': 'req-3edc2356-be91-4ba7-ac40-723b20961a1f'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:35.569916 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-3edc2356-be91-4ba7-ac40-723b20961a1f admin admin] The token cache is being invalidate because project 94689412ae164432a18b7444b74c2bae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:31:35.572760 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-3edc2356-be91-4ba7-ac40-723b20961a1f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:35.582968 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-3edc2356-be91-4ba7-ac40-723b20961a1f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:35.587482 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 889/1778] 10.222.0.102 () {58 vars in 1238 bytes} [Thu Sep 1 17:31:35 2022] DELETE /identity/v3/projects/94689412ae164432a18b7444b74c2bae => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:35.592819 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385effe0-2a63-4d51-99b9-81d63d161227 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:35.593106 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385effe0-2a63-4d51-99b9-81d63d161227 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:35.593399 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385effe0-2a63-4d51-99b9-81d63d161227 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:35.594882 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 890/1779] 10.222.0.102 () {54 vars in 926 bytes} [Thu Sep 1 17:31:35 2022] GET /identity => generated 271 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:31:35.595848 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab3716a-eaf6-4f74-bd8d-dab421434484 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:35.596224 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab3716a-eaf6-4f74-bd8d-dab421434484 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:35.596757 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab3716a-eaf6-4f74-bd8d-dab421434484 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:35.627760 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-2ab3716a-eaf6-4f74-bd8d-dab421434484 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:35.628325 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:35.629383 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-2ab3716a-eaf6-4f74-bd8d-dab421434484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:35.655003 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd09be63-94a6-4cdc-8053-48f99cb56233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:35.656672 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd09be63-94a6-4cdc-8053-48f99cb56233 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:35.657380 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd09be63-94a6-4cdc-8053-48f99cb56233 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:35.658617 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd09be63-94a6-4cdc-8053-48f99cb56233 admin admin] PATH_INFO: `/v3/projects/577f304fbcfb4e01ab66bd4a62d30d8e` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:35.659829 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd09be63-94a6-4cdc-8053-48f99cb56233 admin admin] RBAC: Authorizing `identity:delete_project(project_id=577f304fbcfb4e01ab66bd4a62d30d8e)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:35.662013 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd09be63-94a6-4cdc-8053-48f99cb56233 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:35.728085 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 891/1780] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:35 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:35.738269 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-bd09be63-94a6-4cdc-8053-48f99cb56233 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '577f304fbcfb4e01ab66bd4a62d30d8e', 'request_id': 'req-bd09be63-94a6-4cdc-8053-48f99cb56233'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:35.738829 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0766a5e6-e7d3-402b-a858-5714d7bef0e7 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:35.739121 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0766a5e6-e7d3-402b-a858-5714d7bef0e7 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:35.739394 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0766a5e6-e7d3-402b-a858-5714d7bef0e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:35.746820 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-bd09be63-94a6-4cdc-8053-48f99cb56233 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '577f304fbcfb4e01ab66bd4a62d30d8e', 'request_id': 'req-bd09be63-94a6-4cdc-8053-48f99cb56233'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:35.754746 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-bd09be63-94a6-4cdc-8053-48f99cb56233 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '577f304fbcfb4e01ab66bd4a62d30d8e', 'request_id': 'req-bd09be63-94a6-4cdc-8053-48f99cb56233'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:35.760265 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-0766a5e6-e7d3-402b-a858-5714d7bef0e7 None None] MFA Rules not processed for user `5f8293316baf4774b62fc11d55d50865`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:35.761674 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-0766a5e6-e7d3-402b-a858-5714d7bef0e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:35.765796 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-bd09be63-94a6-4cdc-8053-48f99cb56233 admin admin] The token cache is being invalidate because project 577f304fbcfb4e01ab66bd4a62d30d8e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:31:35.766200 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-bd09be63-94a6-4cdc-8053-48f99cb56233 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:35.766913 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-bd09be63-94a6-4cdc-8053-48f99cb56233 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:35.768586 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 890/1781] 10.222.0.102 () {58 vars in 1238 bytes} [Thu Sep 1 17:31:35 2022] DELETE /identity/v3/projects/577f304fbcfb4e01ab66bd4a62d30d8e => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:31:35.769479 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd09be63-94a6-4cdc-8053-48f99cb56233 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:35.770768 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-bd09be63-94a6-4cdc-8053-48f99cb56233 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:35.838546 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 892/1782] 10.222.0.102 () {58 vars in 1023 bytes} [Thu Sep 1 17:31:35 2022] POST /identity/v3/auth/tokens => generated 3382 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:35.840259 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98690dcd-db24-429b-aec8-e18339752a4b None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:35.840554 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98690dcd-db24-429b-aec8-e18339752a4b None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:35.840891 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98690dcd-db24-429b-aec8-e18339752a4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:35.842079 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd09be63-94a6-4cdc-8053-48f99cb56233 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:35.849594 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b79485c-947c-43d0-99a5-0e176072c916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:35.850761 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b79485c-947c-43d0-99a5-0e176072c916 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:35.852238 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b79485c-947c-43d0-99a5-0e176072c916 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:35.852521 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b79485c-947c-43d0-99a5-0e176072c916 admin admin] PATH_INFO: `/v3/projects/546c57ccc877473c99a212984c372f51` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:35.853905 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b79485c-947c-43d0-99a5-0e176072c916 admin admin] RBAC: Authorizing `identity:delete_project(project_id=546c57ccc877473c99a212984c372f51)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:35.855848 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b79485c-947c-43d0-99a5-0e176072c916 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:35.865961 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-98690dcd-db24-429b-aec8-e18339752a4b None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:35.867175 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-98690dcd-db24-429b-aec8-e18339752a4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:35.875959 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 893/1783] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:35 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:35.877463 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54203a9-5c84-4dad-af15-9405b36813cb None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:35.877562 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54203a9-5c84-4dad-af15-9405b36813cb None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:35.877825 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54203a9-5c84-4dad-af15-9405b36813cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:35.897605 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-b54203a9-5c84-4dad-af15-9405b36813cb None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:35.898800 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-b54203a9-5c84-4dad-af15-9405b36813cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:35.913910 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-6b79485c-947c-43d0-99a5-0e176072c916 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '546c57ccc877473c99a212984c372f51', 'request_id': 'req-6b79485c-947c-43d0-99a5-0e176072c916'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:35.923248 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-6b79485c-947c-43d0-99a5-0e176072c916 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '546c57ccc877473c99a212984c372f51', 'request_id': 'req-6b79485c-947c-43d0-99a5-0e176072c916'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:35.929678 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:35.934587 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-6b79485c-947c-43d0-99a5-0e176072c916 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '546c57ccc877473c99a212984c372f51', 'request_id': 'req-6b79485c-947c-43d0-99a5-0e176072c916'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:35.946892 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-6b79485c-947c-43d0-99a5-0e176072c916 admin admin] The token cache is being invalidate because project 546c57ccc877473c99a212984c372f51 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:31:35.947261 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-6b79485c-947c-43d0-99a5-0e176072c916 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:35.948011 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-6b79485c-947c-43d0-99a5-0e176072c916 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:35.949709 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 891/1784] 10.222.0.102 () {58 vars in 1238 bytes} [Thu Sep 1 17:31:35 2022] DELETE /identity/v3/projects/546c57ccc877473c99a212984c372f51 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:35.950841 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b79485c-947c-43d0-99a5-0e176072c916 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:35.952274 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-6b79485c-947c-43d0-99a5-0e176072c916 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:35.965908 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:35.968228 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 894/1785] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:35 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:35.969239 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54203a9-5c84-4dad-af15-9405b36813cb None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:35.970842 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-b54203a9-5c84-4dad-af15-9405b36813cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:35.984830 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54203a9-5c84-4dad-af15-9405b36813cb None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:35.989837 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c635a09-5e4e-4c64-b037-1907b8e89591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:35.990440 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c635a09-5e4e-4c64-b037-1907b8e89591 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:35.990719 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c635a09-5e4e-4c64-b037-1907b8e89591 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:35.990991 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c635a09-5e4e-4c64-b037-1907b8e89591 admin admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:35.991644 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c635a09-5e4e-4c64-b037-1907b8e89591 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:31:35.991948 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c635a09-5e4e-4c64-b037-1907b8e89591 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:35.993532 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c635a09-5e4e-4c64-b037-1907b8e89591 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:36.005411 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 895/1786] 10.222.0.102 () {58 vars in 1158 bytes} [Thu Sep 1 17:31:35 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:31:36.006756 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca53c2f-ebda-4452-9dd3-c4fcbe13378e None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:36.007042 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca53c2f-ebda-4452-9dd3-c4fcbe13378e None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:36.007314 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca53c2f-ebda-4452-9dd3-c4fcbe13378e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:36.025964 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-6ca53c2f-ebda-4452-9dd3-c4fcbe13378e None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:36.027153 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-6ca53c2f-ebda-4452-9dd3-c4fcbe13378e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:36.034167 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b79485c-947c-43d0-99a5-0e176072c916 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:36.035928 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 896/1787] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:36 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:36.037115 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dcc3a9-91af-4fac-9304-4f3376c6aec5 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:36.037434 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dcc3a9-91af-4fac-9304-4f3376c6aec5 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:36.037712 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dcc3a9-91af-4fac-9304-4f3376c6aec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:36.039812 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765d68ae-003c-4dba-9869-0d4a3eff3e2e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f8293316baf4774b62fc11d55d50865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:36.040525 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765d68ae-003c-4dba-9869-0d4a3eff3e2e service nova] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:36.040810 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765d68ae-003c-4dba-9869-0d4a3eff3e2e service nova] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:36.041081 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765d68ae-003c-4dba-9869-0d4a3eff3e2e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:36.042880 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-765d68ae-003c-4dba-9869-0d4a3eff3e2e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:36.055966 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765d68ae-003c-4dba-9869-0d4a3eff3e2e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:36.057466 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-e2dcc3a9-91af-4fac-9304-4f3376c6aec5 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:36.057721 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765d68ae-003c-4dba-9869-0d4a3eff3e2e service nova] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:36.058642 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-e2dcc3a9-91af-4fac-9304-4f3376c6aec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:36.066888 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 897/1788] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:36 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:36.067789 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dcc3a9-91af-4fac-9304-4f3376c6aec5 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:36.072957 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 892/1789] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:31:35 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:31:36.075103 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909812f0-f4cc-4d2d-ba43-3b554403ef6f None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:36.075386 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909812f0-f4cc-4d2d-ba43-3b554403ef6f None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:36.075944 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909812f0-f4cc-4d2d-ba43-3b554403ef6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:36.078026 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dcc3a9-91af-4fac-9304-4f3376c6aec5 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:36.083143 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0131b17e-265f-4d81-9c4f-6fc7a21f309b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:36.083813 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0131b17e-265f-4d81-9c4f-6fc7a21f309b admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:36.084165 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0131b17e-265f-4d81-9c4f-6fc7a21f309b admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:36.084539 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0131b17e-265f-4d81-9c4f-6fc7a21f309b admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:36.085235 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0131b17e-265f-4d81-9c4f-6fc7a21f309b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:36.087420 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0131b17e-265f-4d81-9c4f-6fc7a21f309b admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:36.099337 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-909812f0-f4cc-4d2d-ba43-3b554403ef6f None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:36.100944 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-909812f0-f4cc-4d2d-ba43-3b554403ef6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:36.114540 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 893/1790] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:36 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:36.114540 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909812f0-f4cc-4d2d-ba43-3b554403ef6f None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:36.118516 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 898/1791] 10.222.0.102 () {60 vars in 1158 bytes} [Thu Sep 1 17:31:36 2022] POST /identity/v3/projects => generated 374 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:31:36.119412 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0131b17e-265f-4d81-9c4f-6fc7a21f309b admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:36.125772 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909812f0-f4cc-4d2d-ba43-3b554403ef6f None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:36.188146 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0131b17e-265f-4d81-9c4f-6fc7a21f309b admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:36.188364 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0bbfb4-572f-443e-8618-9ac05aff0e84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:36.188978 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0bbfb4-572f-443e-8618-9ac05aff0e84 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:36.189259 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0bbfb4-572f-443e-8618-9ac05aff0e84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:36.189565 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0bbfb4-572f-443e-8618-9ac05aff0e84 admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:36.190212 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0bbfb4-572f-443e-8618-9ac05aff0e84 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:31:36.190518 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0bbfb4-572f-443e-8618-9ac05aff0e84 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:36.192126 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0bbfb4-572f-443e-8618-9ac05aff0e84 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:36.192834 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeddae3-9619-4645-a4e0-63f1622db9a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:36.193466 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeddae3-9619-4645-a4e0-63f1622db9a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:36.193748 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeddae3-9619-4645-a4e0-63f1622db9a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:36.194020 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeddae3-9619-4645-a4e0-63f1622db9a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:36.194665 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caeddae3-9619-4645-a4e0-63f1622db9a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:31:36.194967 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caeddae3-9619-4645-a4e0-63f1622db9a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:36.196545 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caeddae3-9619-4645-a4e0-63f1622db9a0 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:36.204627 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 894/1792] 10.222.0.102 () {58 vars in 1158 bytes} [Thu Sep 1 17:31:36 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:31:36.205591 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0bbfb4-572f-443e-8618-9ac05aff0e84 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:36.209901 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 899/1793] 10.222.0.102 () {58 vars in 1158 bytes} [Thu Sep 1 17:31:36 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:31:36.210742 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeddae3-9619-4645-a4e0-63f1622db9a0 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:36.220073 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0bbfb4-572f-443e-8618-9ac05aff0e84 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:36.221119 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeddae3-9619-4645-a4e0-63f1622db9a0 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:36.225478 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683613cc-124c-4673-a465-524ad17003ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:36.226086 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683613cc-124c-4673-a465-524ad17003ce admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:36.226370 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683613cc-124c-4673-a465-524ad17003ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:36.226474 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41d481c-7f23-40a8-b8e8-eb78307e5329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:36.228085 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683613cc-124c-4673-a465-524ad17003ce admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:36.228205 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41d481c-7f23-40a8-b8e8-eb78307e5329 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:36.228300 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683613cc-124c-4673-a465-524ad17003ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:36.228375 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41d481c-7f23-40a8-b8e8-eb78307e5329 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:36.228375 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41d481c-7f23-40a8-b8e8-eb78307e5329 admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:36.228515 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41d481c-7f23-40a8-b8e8-eb78307e5329 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:36.229251 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683613cc-124c-4673-a465-524ad17003ce admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:36.230535 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41d481c-7f23-40a8-b8e8-eb78307e5329 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:36.251313 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 895/1794] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:31:36 2022] POST /identity/v3/users => generated 288 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:31:36.252422 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683613cc-124c-4673-a465-524ad17003ce admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:36.264439 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683613cc-124c-4673-a465-524ad17003ce admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:36.268943 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 900/1795] 10.222.0.102 () {60 vars in 1158 bytes} [Thu Sep 1 17:31:36 2022] POST /identity/v3/projects => generated 378 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:31:36.270899 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6388959-23e8-4fdc-a1cc-d98baee4ae5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:36.270899 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6388959-23e8-4fdc-a1cc-d98baee4ae5f admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:36.270899 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6388959-23e8-4fdc-a1cc-d98baee4ae5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:36.270899 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6388959-23e8-4fdc-a1cc-d98baee4ae5f admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:36.271595 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6388959-23e8-4fdc-a1cc-d98baee4ae5f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:36.272923 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41d481c-7f23-40a8-b8e8-eb78307e5329 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:36.273511 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6388959-23e8-4fdc-a1cc-d98baee4ae5f admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:36.304440 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 896/1796] 10.222.0.102 () {60 vars in 1158 bytes} [Thu Sep 1 17:31:36 2022] POST /identity/v3/projects => generated 386 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:31:36.305863 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6388959-23e8-4fdc-a1cc-d98baee4ae5f admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:36.341222 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:36.347166 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41d481c-7f23-40a8-b8e8-eb78307e5329 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:36.360541 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:36.381920 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6388959-23e8-4fdc-a1cc-d98baee4ae5f admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:36.387667 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adf6bda-8888-4ae7-abbb-27b236fe3a1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:36.389150 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adf6bda-8888-4ae7-abbb-27b236fe3a1c admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:36.389697 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adf6bda-8888-4ae7-abbb-27b236fe3a1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:36.390144 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adf6bda-8888-4ae7-abbb-27b236fe3a1c admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:36.390970 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adf6bda-8888-4ae7-abbb-27b236fe3a1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:36.393174 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adf6bda-8888-4ae7-abbb-27b236fe3a1c admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:36.417914 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 897/1797] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:31:36 2022] POST /identity/v3/users => generated 290 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:31:36.418852 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adf6bda-8888-4ae7-abbb-27b236fe3a1c admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:36.428991 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adf6bda-8888-4ae7-abbb-27b236fe3a1c admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:36.434139 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092c503e-340b-43da-b856-6db045de9e74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:36.434857 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092c503e-340b-43da-b856-6db045de9e74 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:36.435135 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092c503e-340b-43da-b856-6db045de9e74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:36.435443 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092c503e-340b-43da-b856-6db045de9e74 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:36.436078 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092c503e-340b-43da-b856-6db045de9e74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:36.438044 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092c503e-340b-43da-b856-6db045de9e74 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:36.441641 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeb2ec9-8c8c-4704-b40c-047f4350262e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:36.442251 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeb2ec9-8c8c-4704-b40c-047f4350262e admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:36.442530 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeb2ec9-8c8c-4704-b40c-047f4350262e admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:36.442801 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeb2ec9-8c8c-4704-b40c-047f4350262e admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:36.443448 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faeb2ec9-8c8c-4704-b40c-047f4350262e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:36.445064 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faeb2ec9-8c8c-4704-b40c-047f4350262e admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:36.458834 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 901/1798] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:31:36 2022] GET /identity/v3/roles => generated 1427 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:31:36.462579 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 898/1799] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:31:36 2022] POST /identity/v3/users => generated 294 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:31:36.462702 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeb2ec9-8c8c-4704-b40c-047f4350262e admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:36.463823 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092c503e-340b-43da-b856-6db045de9e74 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:36.475413 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeb2ec9-8c8c-4704-b40c-047f4350262e admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:36.476697 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092c503e-340b-43da-b856-6db045de9e74 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:36.481033 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42396e5-8451-4f0f-9dcb-1e6b976d525e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:36.481255 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f414d5d8-54f8-4f93-9bfd-9af7db15170c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:36.481679 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42396e5-8451-4f0f-9dcb-1e6b976d525e admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:36.481896 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f414d5d8-54f8-4f93-9bfd-9af7db15170c admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:36.481991 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42396e5-8451-4f0f-9dcb-1e6b976d525e admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:36.482175 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f414d5d8-54f8-4f93-9bfd-9af7db15170c admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:36.482267 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42396e5-8451-4f0f-9dcb-1e6b976d525e admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:36.482480 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f414d5d8-54f8-4f93-9bfd-9af7db15170c admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:36.483518 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42396e5-8451-4f0f-9dcb-1e6b976d525e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:36.483631 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f414d5d8-54f8-4f93-9bfd-9af7db15170c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:36.484506 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42396e5-8451-4f0f-9dcb-1e6b976d525e admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:36.484784 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f414d5d8-54f8-4f93-9bfd-9af7db15170c admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:36.495413 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 902/1800] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:31:36 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:31:36.496345 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f414d5d8-54f8-4f93-9bfd-9af7db15170c admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:36.500418 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 899/1801] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:31:36 2022] GET /identity/v3/roles => generated 1427 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:31:36.501228 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42396e5-8451-4f0f-9dcb-1e6b976d525e admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:36.508900 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f414d5d8-54f8-4f93-9bfd-9af7db15170c admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:36.511488 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42396e5-8451-4f0f-9dcb-1e6b976d525e admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:36.514607 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2670f17-8cf0-445e-b566-50f235dc288a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:36.515243 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2670f17-8cf0-445e-b566-50f235dc288a admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:36.515706 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2670f17-8cf0-445e-b566-50f235dc288a admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:36.515825 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2670f17-8cf0-445e-b566-50f235dc288a admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:36.516391 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2670f17-8cf0-445e-b566-50f235dc288a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:36.517370 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d492ab-6688-4265-ab17-82ab45c570d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:36.517938 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d492ab-6688-4265-ab17-82ab45c570d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:36.518200 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2670f17-8cf0-445e-b566-50f235dc288a admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:36.518297 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d492ab-6688-4265-ab17-82ab45c570d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:36.518480 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d492ab-6688-4265-ab17-82ab45c570d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:36.519115 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d492ab-6688-4265-ab17-82ab45c570d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:36.520695 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d492ab-6688-4265-ab17-82ab45c570d6 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:36.530922 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 903/1802] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:31:36 2022] GET /identity/v3/roles => generated 1427 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:31:36.532203 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2670f17-8cf0-445e-b566-50f235dc288a admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:36.533382 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 900/1803] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:31:36 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:31:36.536253 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d492ab-6688-4265-ab17-82ab45c570d6 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:36.544312 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2670f17-8cf0-445e-b566-50f235dc288a admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:36.549233 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d492ab-6688-4265-ab17-82ab45c570d6 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:36.549964 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f3b5dc-2396-4e93-b471-6aa3773e1e9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:36.550833 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f3b5dc-2396-4e93-b471-6aa3773e1e9d admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:36.551311 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f3b5dc-2396-4e93-b471-6aa3773e1e9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:36.551418 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f3b5dc-2396-4e93-b471-6aa3773e1e9d admin admin] PATH_INFO: `/v3/projects/2f1b4194551c4d46bf4efd2d8eebe056/users/1013beff7bfc467b8ce3b57dd52aec07/roles/b75cb360925e4ba5a65f14965c103ffd` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:36.554431 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86940ffe-87a8-4770-b37a-c009a545d135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:36.555447 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86940ffe-87a8-4770-b37a-c009a545d135 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:36.555447 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86940ffe-87a8-4770-b37a-c009a545d135 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:36.555626 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86940ffe-87a8-4770-b37a-c009a545d135 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:36.556361 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86940ffe-87a8-4770-b37a-c009a545d135 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:36.557981 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86940ffe-87a8-4770-b37a-c009a545d135 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:36.570126 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 901/1804] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:31:36 2022] GET /identity/v3/roles => generated 1427 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 17:31:36.570126 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86940ffe-87a8-4770-b37a-c009a545d135 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:36.577296 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f3b5dc-2396-4e93-b471-6aa3773e1e9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f1b4194551c4d46bf4efd2d8eebe056, user_id=1013beff7bfc467b8ce3b57dd52aec07, role_id=b75cb360925e4ba5a65f14965c103ffd)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:36.580046 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f3b5dc-2396-4e93-b471-6aa3773e1e9d admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:36.581620 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86940ffe-87a8-4770-b37a-c009a545d135 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:36.587936 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981b03e3-f149-4ab6-878e-ee927ca33cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:36.588757 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981b03e3-f149-4ab6-878e-ee927ca33cfc admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:36.589215 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981b03e3-f149-4ab6-878e-ee927ca33cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:36.589525 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981b03e3-f149-4ab6-878e-ee927ca33cfc admin admin] PATH_INFO: `/v3/projects/57c81b82480a4baa97c0530229ac8e36/users/82afc1385f4a4c219ef6de0c2bdfed05/roles/b75cb360925e4ba5a65f14965c103ffd` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:36.595953 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 904/1805] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:31:36 2022] PUT /identity/v3/projects/2f1b4194551c4d46bf4efd2d8eebe056/users/1013beff7bfc467b8ce3b57dd52aec07/roles/b75cb360925e4ba5a65f14965c103ffd => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:36.601522 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f3b5dc-2396-4e93-b471-6aa3773e1e9d admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:36.605976 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981b03e3-f149-4ab6-878e-ee927ca33cfc admin admin] RBAC: Authorizing `identity:create_grant(project_id=57c81b82480a4baa97c0530229ac8e36, user_id=82afc1385f4a4c219ef6de0c2bdfed05, role_id=b75cb360925e4ba5a65f14965c103ffd)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:36.608990 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981b03e3-f149-4ab6-878e-ee927ca33cfc admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:36.621995 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 902/1806] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:31:36 2022] PUT /identity/v3/projects/57c81b82480a4baa97c0530229ac8e36/users/82afc1385f4a4c219ef6de0c2bdfed05/roles/b75cb360925e4ba5a65f14965c103ffd => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:36.623743 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13febd09-46f1-483e-81d4-a268f8858ed7 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:36.624032 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13febd09-46f1-483e-81d4-a268f8858ed7 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:36.624309 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13febd09-46f1-483e-81d4-a268f8858ed7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:36.653230 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-13febd09-46f1-483e-81d4-a268f8858ed7 None None] MFA Rules not processed for user `1013beff7bfc467b8ce3b57dd52aec07`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:36.654518 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-13febd09-46f1-483e-81d4-a268f8858ed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:36.666531 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f3b5dc-2396-4e93-b471-6aa3773e1e9d admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:36.726591 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 903/1807] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:36 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:36.726777 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7336926-3142-48a8-a0db-8c70e9614442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:36.727518 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7336926-3142-48a8-a0db-8c70e9614442 admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:36.727801 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7336926-3142-48a8-a0db-8c70e9614442 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:36.728072 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7336926-3142-48a8-a0db-8c70e9614442 admin admin] PATH_INFO: `/v3/projects/9f61bc7db6a24f5c990e875a5fed61e3/users/0d38c1bdca814dbf8a61184a11074575/roles/b75cb360925e4ba5a65f14965c103ffd` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:36.728203 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f3cc00-de9f-4f78-86dc-03704f0c46a1 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:36.728478 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f3cc00-de9f-4f78-86dc-03704f0c46a1 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:36.728754 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f3cc00-de9f-4f78-86dc-03704f0c46a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:36.732015 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:36.750262 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7336926-3142-48a8-a0db-8c70e9614442 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f61bc7db6a24f5c990e875a5fed61e3, user_id=0d38c1bdca814dbf8a61184a11074575, role_id=b75cb360925e4ba5a65f14965c103ffd)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:36.754533 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7336926-3142-48a8-a0db-8c70e9614442 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:36.765055 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-98f3cc00-de9f-4f78-86dc-03704f0c46a1 None None] MFA Rules not processed for user `82afc1385f4a4c219ef6de0c2bdfed05`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:36.766787 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-98f3cc00-de9f-4f78-86dc-03704f0c46a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:36.774055 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 905/1808] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:31:36 2022] PUT /identity/v3/projects/9f61bc7db6a24f5c990e875a5fed61e3/users/0d38c1bdca814dbf8a61184a11074575/roles/b75cb360925e4ba5a65f14965c103ffd => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:36.775900 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7336926-3142-48a8-a0db-8c70e9614442 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:36.841537 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:36.901740 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7336926-3142-48a8-a0db-8c70e9614442 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:36.911512 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 904/1809] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:36 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:36.911512 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3373df2c-ab9f-4637-b983-9df52d0e39e3 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:36.911512 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3373df2c-ab9f-4637-b983-9df52d0e39e3 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:36.911512 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3373df2c-ab9f-4637-b983-9df52d0e39e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:36.912577 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857fd323-64a7-428e-952e-2e1f0bc38c54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:36.913419 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857fd323-64a7-428e-952e-2e1f0bc38c54 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:36.913859 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857fd323-64a7-428e-952e-2e1f0bc38c54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:36.914288 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857fd323-64a7-428e-952e-2e1f0bc38c54 admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:36.915198 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857fd323-64a7-428e-952e-2e1f0bc38c54 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:36.917299 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857fd323-64a7-428e-952e-2e1f0bc38c54 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:36.949553 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-3373df2c-ab9f-4637-b983-9df52d0e39e3 None None] MFA Rules not processed for user `0d38c1bdca814dbf8a61184a11074575`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:36.952053 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-3373df2c-ab9f-4637-b983-9df52d0e39e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:36.963761 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 906/1810] 10.222.0.102 () {60 vars in 1158 bytes} [Thu Sep 1 17:31:36 2022] POST /identity/v3/projects => generated 374 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:31:36.964786 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857fd323-64a7-428e-952e-2e1f0bc38c54 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:37.040528 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 905/1811] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:36 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:37.041643 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3373df2c-ab9f-4637-b983-9df52d0e39e3 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:37.052008 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857fd323-64a7-428e-952e-2e1f0bc38c54 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:37.056495 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9755658-a9f2-482a-ad11-3e0984444d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:37.057108 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9755658-a9f2-482a-ad11-3e0984444d8e admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:37.057297 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3373df2c-ab9f-4637-b983-9df52d0e39e3 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:37.057563 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9755658-a9f2-482a-ad11-3e0984444d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:37.057695 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9755658-a9f2-482a-ad11-3e0984444d8e admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:37.058492 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9755658-a9f2-482a-ad11-3e0984444d8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:37.061340 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9755658-a9f2-482a-ad11-3e0984444d8e admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:37.063409 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9a8a0c-fbdd-4bd8-a047-a82dce2d54e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:37.064181 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9a8a0c-fbdd-4bd8-a047-a82dce2d54e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:37.064569 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9a8a0c-fbdd-4bd8-a047-a82dce2d54e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:37.064937 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9a8a0c-fbdd-4bd8-a047-a82dce2d54e5 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:37.065736 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9a8a0c-fbdd-4bd8-a047-a82dce2d54e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:37.067753 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9a8a0c-fbdd-4bd8-a047-a82dce2d54e5 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:37.095072 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 907/1812] 10.222.0.102 () {60 vars in 1158 bytes} [Thu Sep 1 17:31:36 2022] POST /identity/v3/projects => generated 380 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:31:37.095881 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9755658-a9f2-482a-ad11-3e0984444d8e admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:37.126525 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 906/1813] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:31:37 2022] POST /identity/v3/users => generated 294 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:31:37.127189 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9a8a0c-fbdd-4bd8-a047-a82dce2d54e5 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:37.139363 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9a8a0c-fbdd-4bd8-a047-a82dce2d54e5 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:37.173616 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:37.187260 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9755658-a9f2-482a-ad11-3e0984444d8e admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:37.192028 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15543190-5ded-460b-bcb2-286251fceea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:37.192638 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15543190-5ded-460b-bcb2-286251fceea6 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:37.192927 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15543190-5ded-460b-bcb2-286251fceea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:37.193372 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15543190-5ded-460b-bcb2-286251fceea6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:37.194015 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15543190-5ded-460b-bcb2-286251fceea6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:37.195951 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15543190-5ded-460b-bcb2-286251fceea6 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:37.219893 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ecacc3-6cf0-4c03-9acd-0e1caaba586e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:37.220513 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ecacc3-6cf0-4c03-9acd-0e1caaba586e admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:37.220796 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ecacc3-6cf0-4c03-9acd-0e1caaba586e admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:37.221070 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ecacc3-6cf0-4c03-9acd-0e1caaba586e admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:37.221993 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ecacc3-6cf0-4c03-9acd-0e1caaba586e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:37.222921 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 908/1814] 10.222.0.102 () {60 vars in 1158 bytes} [Thu Sep 1 17:31:37 2022] POST /identity/v3/projects => generated 388 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:31:37.223625 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15543190-5ded-460b-bcb2-286251fceea6 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:37.224110 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ecacc3-6cf0-4c03-9acd-0e1caaba586e admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:37.245851 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 907/1815] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:31:37 2022] POST /identity/v3/users => generated 291 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:31:37.250315 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ecacc3-6cf0-4c03-9acd-0e1caaba586e admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:37.260424 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ecacc3-6cf0-4c03-9acd-0e1caaba586e admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:37.300777 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15543190-5ded-460b-bcb2-286251fceea6 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:37.307197 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522447d4-e976-4731-8491-b492be5b63be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:37.307947 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522447d4-e976-4731-8491-b492be5b63be admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:37.308237 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522447d4-e976-4731-8491-b492be5b63be admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:37.308526 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522447d4-e976-4731-8491-b492be5b63be admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:37.309222 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522447d4-e976-4731-8491-b492be5b63be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:37.311664 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522447d4-e976-4731-8491-b492be5b63be admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:37.314247 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:37.327727 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 909/1816] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:31:37 2022] GET /identity/v3/roles => generated 1427 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:31:37.328663 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522447d4-e976-4731-8491-b492be5b63be admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:37.337077 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20c4fb6-c665-4745-ab84-32348cb2ec82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:37.337841 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20c4fb6-c665-4745-ab84-32348cb2ec82 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:37.338198 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20c4fb6-c665-4745-ab84-32348cb2ec82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:37.338777 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20c4fb6-c665-4745-ab84-32348cb2ec82 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:37.339604 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20c4fb6-c665-4745-ab84-32348cb2ec82 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:37.340957 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522447d4-e976-4731-8491-b492be5b63be admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:37.342587 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20c4fb6-c665-4745-ab84-32348cb2ec82 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:37.348199 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6027759c-a3a0-4ea7-abdf-164d0262b1e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:37.348780 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6027759c-a3a0-4ea7-abdf-164d0262b1e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:37.349146 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6027759c-a3a0-4ea7-abdf-164d0262b1e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:37.349530 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6027759c-a3a0-4ea7-abdf-164d0262b1e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:37.350284 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6027759c-a3a0-4ea7-abdf-164d0262b1e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:37.352352 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6027759c-a3a0-4ea7-abdf-164d0262b1e8 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:37.372404 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 910/1817] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:31:37 2022] GET /identity/v3/roles => generated 1427 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:31:37.372404 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6027759c-a3a0-4ea7-abdf-164d0262b1e8 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:37.372677 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 908/1818] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:31:37 2022] POST /identity/v3/users => generated 295 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:31:37.375231 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20c4fb6-c665-4745-ab84-32348cb2ec82 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:37.383106 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6027759c-a3a0-4ea7-abdf-164d0262b1e8 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:37.386867 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20c4fb6-c665-4745-ab84-32348cb2ec82 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:37.388148 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd37fb14-84d8-4ae2-ae65-5c94dbc5793b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:37.388893 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd37fb14-84d8-4ae2-ae65-5c94dbc5793b admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:37.389247 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd37fb14-84d8-4ae2-ae65-5c94dbc5793b admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:37.389578 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd37fb14-84d8-4ae2-ae65-5c94dbc5793b admin admin] PATH_INFO: `/v3/projects/1c94345144394c9aba353fbc3b0ba56d/users/408767787bbd4e8eb07f1af315ab8e45/roles/36577a6f46d642d4b81805ede9872490` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:37.393460 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e57a5cc-4914-49ac-900f-18eac4b69e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:37.394122 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e57a5cc-4914-49ac-900f-18eac4b69e16 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:37.394409 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e57a5cc-4914-49ac-900f-18eac4b69e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:37.394686 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e57a5cc-4914-49ac-900f-18eac4b69e16 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:37.395333 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e57a5cc-4914-49ac-900f-18eac4b69e16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:37.397596 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e57a5cc-4914-49ac-900f-18eac4b69e16 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:37.410956 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd37fb14-84d8-4ae2-ae65-5c94dbc5793b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c94345144394c9aba353fbc3b0ba56d, user_id=408767787bbd4e8eb07f1af315ab8e45, role_id=36577a6f46d642d4b81805ede9872490)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:37.413385 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 909/1819] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:31:37 2022] GET /identity/v3/roles => generated 1427 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:31:37.414390 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e57a5cc-4914-49ac-900f-18eac4b69e16 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:37.417623 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd37fb14-84d8-4ae2-ae65-5c94dbc5793b admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:37.428605 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e57a5cc-4914-49ac-900f-18eac4b69e16 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:37.457369 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 911/1820] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:31:37 2022] PUT /identity/v3/projects/1c94345144394c9aba353fbc3b0ba56d/users/408767787bbd4e8eb07f1af315ab8e45/roles/36577a6f46d642d4b81805ede9872490 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:37.457369 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd37fb14-84d8-4ae2-ae65-5c94dbc5793b admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:37.514666 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e58787-dec6-4662-9043-79b0dab8e798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:37.515691 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e58787-dec6-4662-9043-79b0dab8e798 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:37.516121 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e58787-dec6-4662-9043-79b0dab8e798 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:37.516782 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e58787-dec6-4662-9043-79b0dab8e798 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:37.517648 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e58787-dec6-4662-9043-79b0dab8e798 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:37.520004 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e58787-dec6-4662-9043-79b0dab8e798 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:37.537751 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 910/1821] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:31:37 2022] GET /identity/v3/roles => generated 1427 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:31:37.538929 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e58787-dec6-4662-9043-79b0dab8e798 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:37.554746 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e58787-dec6-4662-9043-79b0dab8e798 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:37.566607 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fd7953-ed1c-4334-897c-44e518eeddf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:37.567787 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fd7953-ed1c-4334-897c-44e518eeddf1 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:37.568149 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fd7953-ed1c-4334-897c-44e518eeddf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:37.568473 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fd7953-ed1c-4334-897c-44e518eeddf1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:37.569236 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fd7953-ed1c-4334-897c-44e518eeddf1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:37.572933 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fd7953-ed1c-4334-897c-44e518eeddf1 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:37.598336 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 911/1822] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:31:37 2022] GET /identity/v3/roles => generated 1427 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:31:37.599247 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fd7953-ed1c-4334-897c-44e518eeddf1 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:37.619327 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fd7953-ed1c-4334-897c-44e518eeddf1 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:37.626113 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f8c7af-77c1-4694-b443-7e4df640a7cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:37.626911 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f8c7af-77c1-4694-b443-7e4df640a7cd admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:37.627466 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f8c7af-77c1-4694-b443-7e4df640a7cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:37.627950 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f8c7af-77c1-4694-b443-7e4df640a7cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:37.628775 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f8c7af-77c1-4694-b443-7e4df640a7cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:37.630975 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f8c7af-77c1-4694-b443-7e4df640a7cd admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:37.635353 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd37fb14-84d8-4ae2-ae65-5c94dbc5793b admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:37.643792 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 912/1823] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:31:37 2022] GET /identity/v3/roles => generated 1427 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:31:37.657972 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f8c7af-77c1-4694-b443-7e4df640a7cd admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:37.663587 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564f5659-29c4-4c9c-8bfa-190601dc8eb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:37.664716 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564f5659-29c4-4c9c-8bfa-190601dc8eb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:37.665345 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564f5659-29c4-4c9c-8bfa-190601dc8eb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:37.665778 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564f5659-29c4-4c9c-8bfa-190601dc8eb9 admin admin] PATH_INFO: `/v3/projects/086fe57f9151486fbd9313f2ce2560ef/users/9558cef7df8f4f478666ae56e0ac6169/roles/b75cb360925e4ba5a65f14965c103ffd` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:37.671366 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f8c7af-77c1-4694-b443-7e4df640a7cd admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:37.678193 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3d20eb-00c2-4bb7-b410-58a5c2c2ff71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:37.679356 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3d20eb-00c2-4bb7-b410-58a5c2c2ff71 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:37.679836 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3d20eb-00c2-4bb7-b410-58a5c2c2ff71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:37.680375 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3d20eb-00c2-4bb7-b410-58a5c2c2ff71 admin admin] PATH_INFO: `/v3/projects/1c94345144394c9aba353fbc3b0ba56d/users/408767787bbd4e8eb07f1af315ab8e45/roles/36577a6f46d642d4b81805ede9872490` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:37.683503 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3d20eb-00c2-4bb7-b410-58a5c2c2ff71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c94345144394c9aba353fbc3b0ba56d, user_id=408767787bbd4e8eb07f1af315ab8e45, role_id=36577a6f46d642d4b81805ede9872490)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:37.688911 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3d20eb-00c2-4bb7-b410-58a5c2c2ff71 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:37.705971 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564f5659-29c4-4c9c-8bfa-190601dc8eb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=086fe57f9151486fbd9313f2ce2560ef, user_id=9558cef7df8f4f478666ae56e0ac6169, role_id=b75cb360925e4ba5a65f14965c103ffd)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:37.708982 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564f5659-29c4-4c9c-8bfa-190601dc8eb9 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:37.716609 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 913/1824] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:31:37 2022] PUT /identity/v3/projects/1c94345144394c9aba353fbc3b0ba56d/users/408767787bbd4e8eb07f1af315ab8e45/roles/36577a6f46d642d4b81805ede9872490 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:37.722723 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3d20eb-00c2-4bb7-b410-58a5c2c2ff71 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:37.764940 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 912/1825] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:31:37 2022] PUT /identity/v3/projects/086fe57f9151486fbd9313f2ce2560ef/users/9558cef7df8f4f478666ae56e0ac6169/roles/b75cb360925e4ba5a65f14965c103ffd => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:37.766517 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5d4ca5-be6d-4461-ac62-70071f178c19 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:37.766949 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5d4ca5-be6d-4461-ac62-70071f178c19 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:37.767319 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5d4ca5-be6d-4461-ac62-70071f178c19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:37.820450 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3d20eb-00c2-4bb7-b410-58a5c2c2ff71 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:37.836093 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-0e5d4ca5-be6d-4461-ac62-70071f178c19 None None] MFA Rules not processed for user `408767787bbd4e8eb07f1af315ab8e45`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:37.841686 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-0e5d4ca5-be6d-4461-ac62-70071f178c19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:37.885110 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:37.913687 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:38.027770 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fac426b-26cb-42a5-a38b-2a01dc775301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:38.028468 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fac426b-26cb-42a5-a38b-2a01dc775301 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:38.028770 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fac426b-26cb-42a5-a38b-2a01dc775301 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:38.029078 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fac426b-26cb-42a5-a38b-2a01dc775301 admin admin] PATH_INFO: `/v3/projects/c37f6b8752bb445cbef2eb2a510aba66/users/ace575ade14541cc8ae4bd03d48acebc/roles/b75cb360925e4ba5a65f14965c103ffd` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:38.066286 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fac426b-26cb-42a5-a38b-2a01dc775301 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c37f6b8752bb445cbef2eb2a510aba66, user_id=ace575ade14541cc8ae4bd03d48acebc, role_id=b75cb360925e4ba5a65f14965c103ffd)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:38.068989 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fac426b-26cb-42a5-a38b-2a01dc775301 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:38.081236 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 913/1826] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:37 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:38.081236 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b83705-eabd-4a15-ba8b-aad6e44a584c None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:38.081236 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b83705-eabd-4a15-ba8b-aad6e44a584c None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:38.081236 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b83705-eabd-4a15-ba8b-aad6e44a584c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:38.104878 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 914/1827] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:31:37 2022] PUT /identity/v3/projects/c37f6b8752bb445cbef2eb2a510aba66/users/ace575ade14541cc8ae4bd03d48acebc/roles/b75cb360925e4ba5a65f14965c103ffd => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:38.118692 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317aee49-61c4-4ea4-9602-d5fe57b82280 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:38.118964 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317aee49-61c4-4ea4-9602-d5fe57b82280 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:38.119169 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317aee49-61c4-4ea4-9602-d5fe57b82280 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:38.128070 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-98b83705-eabd-4a15-ba8b-aad6e44a584c None None] MFA Rules not processed for user `9558cef7df8f4f478666ae56e0ac6169`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:38.129479 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-98b83705-eabd-4a15-ba8b-aad6e44a584c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:38.201873 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-317aee49-61c4-4ea4-9602-d5fe57b82280 None None] MFA Rules not processed for user `ace575ade14541cc8ae4bd03d48acebc`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:38.203834 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-317aee49-61c4-4ea4-9602-d5fe57b82280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:38.299092 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 914/1828] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:38 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:38.300822 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b83705-eabd-4a15-ba8b-aad6e44a584c None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:38.303067 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-98b83705-eabd-4a15-ba8b-aad6e44a584c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:38.334104 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 915/1829] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:38 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:31:38.334785 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317aee49-61c4-4ea4-9602-d5fe57b82280 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:38.336053 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-317aee49-61c4-4ea4-9602-d5fe57b82280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:38.409645 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b83705-eabd-4a15-ba8b-aad6e44a584c None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:38.413505 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bfda27-122b-4642-b859-1127a2431ff6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '523ed589872a45a09b6a7675a7fb0b75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:38.414119 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bfda27-122b-4642-b859-1127a2431ff6 service cinder] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:38.414399 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bfda27-122b-4642-b859-1127a2431ff6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:38.414670 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bfda27-122b-4642-b859-1127a2431ff6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:38.421381 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317aee49-61c4-4ea4-9602-d5fe57b82280 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:38.427213 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9e90f4-abb9-40e0-82e4-faf816143941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:38.429518 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9e90f4-abb9-40e0-82e4-faf816143941 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:38.430152 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9e90f4-abb9-40e0-82e4-faf816143941 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:38.430646 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9e90f4-abb9-40e0-82e4-faf816143941 admin admin] PATH_INFO: `/v3/users/59f88fd43f564c4abfe06c0faca536cb` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:38.433517 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9e90f4-abb9-40e0-82e4-faf816143941 admin admin] RBAC: Authorizing `identity:delete_user(user_id=59f88fd43f564c4abfe06c0faca536cb)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:38.437138 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9e90f4-abb9-40e0-82e4-faf816143941 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:38.475154 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:38.490950 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bfda27-122b-4642-b859-1127a2431ff6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:38.492520 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bfda27-122b-4642-b859-1127a2431ff6 service cinder] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:38.504144 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 915/1830] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:31:38 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:31:38.505083 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bfda27-122b-4642-b859-1127a2431ff6 service cinder] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:38.507680 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:38.525940 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-7c9e90f4-abb9-40e0-82e4-faf816143941 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59f88fd43f564c4abfe06c0faca536cb'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:38.527076 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bfda27-122b-4642-b859-1127a2431ff6 service cinder] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:38.536948 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a236806a-55c1-49b1-b03f-88849f11c5b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:38.539459 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a236806a-55c1-49b1-b03f-88849f11c5b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:38.540123 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a236806a-55c1-49b1-b03f-88849f11c5b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:38.541113 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a236806a-55c1-49b1-b03f-88849f11c5b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:38.543228 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a236806a-55c1-49b1-b03f-88849f11c5b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:38.549046 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a236806a-55c1-49b1-b03f-88849f11c5b0 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:38.553516 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-7c9e90f4-abb9-40e0-82e4-faf816143941 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59f88fd43f564c4abfe06c0faca536cb'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:38.580370 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-7c9e90f4-abb9-40e0-82e4-faf816143941 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59f88fd43f564c4abfe06c0faca536cb'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:38.582335 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-7c9e90f4-abb9-40e0-82e4-faf816143941 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59f88fd43f564c4abfe06c0faca536cb'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:38.583596 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-7c9e90f4-abb9-40e0-82e4-faf816143941 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59f88fd43f564c4abfe06c0faca536cb'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:38.609863 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 916/1831] 10.222.0.102 () {60 vars in 1158 bytes} [Thu Sep 1 17:31:38 2022] POST /identity/v3/projects => generated 378 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:31:38.611949 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a236806a-55c1-49b1-b03f-88849f11c5b0 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:38.612908 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 916/1832] 10.222.0.102 () {58 vars in 1229 bytes} [Thu Sep 1 17:31:38 2022] DELETE /identity/v3/users/59f88fd43f564c4abfe06c0faca536cb => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:38.613980 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9e90f4-abb9-40e0-82e4-faf816143941 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:38.614920 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-a236806a-55c1-49b1-b03f-88849f11c5b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:38.615641 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-7c9e90f4-abb9-40e0-82e4-faf816143941 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:38.757887 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9e90f4-abb9-40e0-82e4-faf816143941 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:38.764485 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf9b8aa-46a1-4011-9fea-cc562736ba32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:38.765088 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf9b8aa-46a1-4011-9fea-cc562736ba32 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:38.765389 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf9b8aa-46a1-4011-9fea-cc562736ba32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:38.765703 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf9b8aa-46a1-4011-9fea-cc562736ba32 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:38.766397 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf9b8aa-46a1-4011-9fea-cc562736ba32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:38.768565 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf9b8aa-46a1-4011-9fea-cc562736ba32 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:38.799858 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a236806a-55c1-49b1-b03f-88849f11c5b0 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:38.806277 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcef293e-4e8c-4cde-bf54-93e741dc5cc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:38.808011 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 917/1833] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:31:38 2022] POST /identity/v3/users => generated 296 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:31:38.809401 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfffc34a-8a30-4efa-97ab-a23fe8f1dba4 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:38.809685 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfffc34a-8a30-4efa-97ab-a23fe8f1dba4 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:38.809962 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfffc34a-8a30-4efa-97ab-a23fe8f1dba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:38.817195 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcef293e-4e8c-4cde-bf54-93e741dc5cc0 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:38.817195 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcef293e-4e8c-4cde-bf54-93e741dc5cc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:38.817508 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcef293e-4e8c-4cde-bf54-93e741dc5cc0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:38.818139 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcef293e-4e8c-4cde-bf54-93e741dc5cc0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:38.820267 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcef293e-4e8c-4cde-bf54-93e741dc5cc0 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:38.862126 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-cfffc34a-8a30-4efa-97ab-a23fe8f1dba4 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:38.864425 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-cfffc34a-8a30-4efa-97ab-a23fe8f1dba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:38.873676 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 917/1834] 10.222.0.102 () {60 vars in 1158 bytes} [Thu Sep 1 17:31:38 2022] POST /identity/v3/projects => generated 388 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:31:38.873676 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcef293e-4e8c-4cde-bf54-93e741dc5cc0 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:39.085992 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 918/1835] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:38 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:39.086937 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfffc34a-8a30-4efa-97ab-a23fe8f1dba4 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:39.106352 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfffc34a-8a30-4efa-97ab-a23fe8f1dba4 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:39.121510 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87b6807-2205-4b94-81f8-1bb1cbe29390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:39.122595 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87b6807-2205-4b94-81f8-1bb1cbe29390 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:39.123121 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87b6807-2205-4b94-81f8-1bb1cbe29390 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:39.123603 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87b6807-2205-4b94-81f8-1bb1cbe29390 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:39.124730 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcef293e-4e8c-4cde-bf54-93e741dc5cc0 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:39.130484 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87b6807-2205-4b94-81f8-1bb1cbe29390 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:39.144461 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87b6807-2205-4b94-81f8-1bb1cbe29390 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:39.151276 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8170e910-8307-4f4f-83e7-ee9b3d5886ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:39.155912 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8170e910-8307-4f4f-83e7-ee9b3d5886ec admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:39.156255 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8170e910-8307-4f4f-83e7-ee9b3d5886ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:39.156606 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8170e910-8307-4f4f-83e7-ee9b3d5886ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:39.157362 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8170e910-8307-4f4f-83e7-ee9b3d5886ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:39.164206 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8170e910-8307-4f4f-83e7-ee9b3d5886ec admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:39.183007 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 918/1836] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:31:38 2022] GET /identity/v3/roles => generated 1427 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:31:39.191473 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8170e910-8307-4f4f-83e7-ee9b3d5886ec admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:39.193471 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-8170e910-8307-4f4f-83e7-ee9b3d5886ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:39.211519 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 919/1837] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:31:39 2022] POST /identity/v3/users => generated 301 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:31:39.212473 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87b6807-2205-4b94-81f8-1bb1cbe29390 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:39.264509 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87b6807-2205-4b94-81f8-1bb1cbe29390 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:39.285901 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d307bbcc-9a65-4ff3-a8c7-73698a017767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:39.286593 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d307bbcc-9a65-4ff3-a8c7-73698a017767 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:39.286926 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d307bbcc-9a65-4ff3-a8c7-73698a017767 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:39.287203 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d307bbcc-9a65-4ff3-a8c7-73698a017767 admin admin] PATH_INFO: `/v3/projects/ab1ac2c673e845a19f8afa8e5437c47e/users/03a5c87c21744f3c8d06e01ef1839ccb/roles/36577a6f46d642d4b81805ede9872490` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:39.333661 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d307bbcc-9a65-4ff3-a8c7-73698a017767 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab1ac2c673e845a19f8afa8e5437c47e, user_id=03a5c87c21744f3c8d06e01ef1839ccb, role_id=36577a6f46d642d4b81805ede9872490)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:39.339566 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d307bbcc-9a65-4ff3-a8c7-73698a017767 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:39.367389 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 920/1838] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:31:39 2022] PUT /identity/v3/projects/ab1ac2c673e845a19f8afa8e5437c47e/users/03a5c87c21744f3c8d06e01ef1839ccb/roles/36577a6f46d642d4b81805ede9872490 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:39.367389 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d307bbcc-9a65-4ff3-a8c7-73698a017767 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:39.371809 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:39.391073 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=88,keystone:DELETE=4,keystone:INSERT=8 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:39.416761 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8170e910-8307-4f4f-83e7-ee9b3d5886ec admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:39.461283 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d307bbcc-9a65-4ff3-a8c7-73698a017767 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:39.467600 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f4932c-17aa-43b2-9d9f-de9d969207c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:39.468302 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f4932c-17aa-43b2-9d9f-de9d969207c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:39.468595 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f4932c-17aa-43b2-9d9f-de9d969207c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:39.468878 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f4932c-17aa-43b2-9d9f-de9d969207c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:39.469814 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f4932c-17aa-43b2-9d9f-de9d969207c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:39.472029 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f4932c-17aa-43b2-9d9f-de9d969207c9 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:39.487216 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 921/1839] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:31:39 2022] GET /identity/v3/roles => generated 1427 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:31:39.488349 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f4932c-17aa-43b2-9d9f-de9d969207c9 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:39.503654 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f4932c-17aa-43b2-9d9f-de9d969207c9 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:39.519791 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7daa2e-7867-4d49-b1df-1c9c95f47f08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:39.520417 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7daa2e-7867-4d49-b1df-1c9c95f47f08 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:39.520690 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7daa2e-7867-4d49-b1df-1c9c95f47f08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:39.520960 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7daa2e-7867-4d49-b1df-1c9c95f47f08 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:39.521629 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7daa2e-7867-4d49-b1df-1c9c95f47f08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:39.525099 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7daa2e-7867-4d49-b1df-1c9c95f47f08 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:39.548348 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 922/1840] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:31:39 2022] GET /identity/v3/roles => generated 1427 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:31:39.548348 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7daa2e-7867-4d49-b1df-1c9c95f47f08 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:39.548348 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-ed7daa2e-7867-4d49-b1df-1c9c95f47f08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:39.554011 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4977e81a-b5ff-4e10-9845-5b7f0de871be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:39.555610 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4977e81a-b5ff-4e10-9845-5b7f0de871be service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:39.556565 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4977e81a-b5ff-4e10-9845-5b7f0de871be service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:39.557233 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4977e81a-b5ff-4e10-9845-5b7f0de871be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:39.584671 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7daa2e-7867-4d49-b1df-1c9c95f47f08 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:39.584922 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4977e81a-b5ff-4e10-9845-5b7f0de871be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:39.586465 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4977e81a-b5ff-4e10-9845-5b7f0de871be service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:39.601894 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f76d19-baf0-4ad7-83e4-8e248fbfbf6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:39.602799 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f76d19-baf0-4ad7-83e4-8e248fbfbf6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:39.603244 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f76d19-baf0-4ad7-83e4-8e248fbfbf6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:39.603683 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f76d19-baf0-4ad7-83e4-8e248fbfbf6f admin admin] PATH_INFO: `/v3/users/1013beff7bfc467b8ce3b57dd52aec07` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:39.606082 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f76d19-baf0-4ad7-83e4-8e248fbfbf6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=1013beff7bfc467b8ce3b57dd52aec07)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:39.608952 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f76d19-baf0-4ad7-83e4-8e248fbfbf6f admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:39.625797 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 919/1841] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:31:39 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:31:39.630065 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4977e81a-b5ff-4e10-9845-5b7f0de871be service neutron] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:39.665834 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4977e81a-b5ff-4e10-9845-5b7f0de871be service neutron] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:39.679377 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2f8bbb-1f30-4287-8f6e-86ac2488dfea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:39.680829 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2f8bbb-1f30-4287-8f6e-86ac2488dfea admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:39.681156 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2f8bbb-1f30-4287-8f6e-86ac2488dfea admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:39.682020 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2f8bbb-1f30-4287-8f6e-86ac2488dfea admin admin] PATH_INFO: `/v3/projects/d7eeea5e32ce483089aef76cf8097d67/users/278aa08de8ea41aeb477218d757dcd53/roles/36577a6f46d642d4b81805ede9872490` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:39.714476 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-89f76d19-baf0-4ad7-83e4-8e248fbfbf6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1013beff7bfc467b8ce3b57dd52aec07'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:39.730128 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2f8bbb-1f30-4287-8f6e-86ac2488dfea admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7eeea5e32ce483089aef76cf8097d67, user_id=278aa08de8ea41aeb477218d757dcd53, role_id=36577a6f46d642d4b81805ede9872490)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:39.733272 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2f8bbb-1f30-4287-8f6e-86ac2488dfea admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:39.750534 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-89f76d19-baf0-4ad7-83e4-8e248fbfbf6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1013beff7bfc467b8ce3b57dd52aec07'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:39.752019 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 920/1842] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:31:39 2022] PUT /identity/v3/projects/d7eeea5e32ce483089aef76cf8097d67/users/278aa08de8ea41aeb477218d757dcd53/roles/36577a6f46d642d4b81805ede9872490 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:39.752854 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2f8bbb-1f30-4287-8f6e-86ac2488dfea admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:39.770815 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-89f76d19-baf0-4ad7-83e4-8e248fbfbf6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1013beff7bfc467b8ce3b57dd52aec07'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:39.771634 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-89f76d19-baf0-4ad7-83e4-8e248fbfbf6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1013beff7bfc467b8ce3b57dd52aec07'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:39.772521 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-89f76d19-baf0-4ad7-83e4-8e248fbfbf6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1013beff7bfc467b8ce3b57dd52aec07'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:39.801987 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 923/1843] 10.222.0.102 () {58 vars in 1229 bytes} [Thu Sep 1 17:31:39 2022] DELETE /identity/v3/users/1013beff7bfc467b8ce3b57dd52aec07 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:39.803551 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f76d19-baf0-4ad7-83e4-8e248fbfbf6f admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:39.805956 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-89f76d19-baf0-4ad7-83e4-8e248fbfbf6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:39.901795 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2f8bbb-1f30-4287-8f6e-86ac2488dfea admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:39.919627 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f76d19-baf0-4ad7-83e4-8e248fbfbf6f admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:39.926158 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8281d91-ecb4-4830-97ce-31c25f29e931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:39.927366 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8281d91-ecb4-4830-97ce-31c25f29e931 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:39.927890 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8281d91-ecb4-4830-97ce-31c25f29e931 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:39.928310 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8281d91-ecb4-4830-97ce-31c25f29e931 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:39.929490 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8281d91-ecb4-4830-97ce-31c25f29e931 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:39.933431 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8281d91-ecb4-4830-97ce-31c25f29e931 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:39.941936 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:39.953093 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 924/1844] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:31:39 2022] GET /identity/v3/roles => generated 1427 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:31:39.955072 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732c261b-43a1-4ced-a37a-0f5fda54c74a None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:39.955694 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732c261b-43a1-4ced-a37a-0f5fda54c74a None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:39.956254 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732c261b-43a1-4ced-a37a-0f5fda54c74a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:40.003912 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-732c261b-43a1-4ced-a37a-0f5fda54c74a None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:40.005196 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-732c261b-43a1-4ced-a37a-0f5fda54c74a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:40.013752 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 925/1845] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:39 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:40.014653 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732c261b-43a1-4ced-a37a-0f5fda54c74a None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:40.029279 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732c261b-43a1-4ced-a37a-0f5fda54c74a None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:40.045961 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20489bf4-6bf1-4966-84d1-ddeb4d9a5dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:40.047314 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20489bf4-6bf1-4966-84d1-ddeb4d9a5dc9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:40.048105 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20489bf4-6bf1-4966-84d1-ddeb4d9a5dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:40.048426 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20489bf4-6bf1-4966-84d1-ddeb4d9a5dc9 admin admin] PATH_INFO: `/v3/projects/d7eeea5e32ce483089aef76cf8097d67/users/278aa08de8ea41aeb477218d757dcd53/roles/36577a6f46d642d4b81805ede9872490` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:40.050971 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20489bf4-6bf1-4966-84d1-ddeb4d9a5dc9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7eeea5e32ce483089aef76cf8097d67, user_id=278aa08de8ea41aeb477218d757dcd53, role_id=36577a6f46d642d4b81805ede9872490)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:40.055657 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20489bf4-6bf1-4966-84d1-ddeb4d9a5dc9 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:40.067345 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61df274-530e-41fc-b05d-71c085f27f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:40.068940 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61df274-530e-41fc-b05d-71c085f27f20 admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:40.069704 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61df274-530e-41fc-b05d-71c085f27f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:40.070514 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61df274-530e-41fc-b05d-71c085f27f20 admin admin] PATH_INFO: `/v3/projects/ab1ac2c673e845a19f8afa8e5437c47e/users/03a5c87c21744f3c8d06e01ef1839ccb/roles/36577a6f46d642d4b81805ede9872490` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:40.074279 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61df274-530e-41fc-b05d-71c085f27f20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab1ac2c673e845a19f8afa8e5437c47e, user_id=03a5c87c21744f3c8d06e01ef1839ccb, role_id=36577a6f46d642d4b81805ede9872490)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:40.078714 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61df274-530e-41fc-b05d-71c085f27f20 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:40.079999 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 926/1846] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:31:40 2022] PUT /identity/v3/projects/d7eeea5e32ce483089aef76cf8097d67/users/278aa08de8ea41aeb477218d757dcd53/roles/36577a6f46d642d4b81805ede9872490 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:40.081103 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20489bf4-6bf1-4966-84d1-ddeb4d9a5dc9 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:40.083667 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-20489bf4-6bf1-4966-84d1-ddeb4d9a5dc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:40.095973 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 921/1847] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:31:39 2022] PUT /identity/v3/projects/ab1ac2c673e845a19f8afa8e5437c47e/users/03a5c87c21744f3c8d06e01ef1839ccb/roles/36577a6f46d642d4b81805ede9872490 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:40.099322 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51cda72-313c-4171-bc61-4280444582b6 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:40.099822 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51cda72-313c-4171-bc61-4280444582b6 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:40.100288 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51cda72-313c-4171-bc61-4280444582b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:40.141090 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-d51cda72-313c-4171-bc61-4280444582b6 None None] MFA Rules not processed for user `278aa08de8ea41aeb477218d757dcd53`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:40.141090 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-d51cda72-313c-4171-bc61-4280444582b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:40.175213 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:40.222211 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20489bf4-6bf1-4966-84d1-ddeb4d9a5dc9 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:40.312223 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632f26d7-51ab-4585-8b29-250065837eb6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:40.313099 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632f26d7-51ab-4585-8b29-250065837eb6 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:40.313622 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632f26d7-51ab-4585-8b29-250065837eb6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:40.313904 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632f26d7-51ab-4585-8b29-250065837eb6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:40.347086 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 922/1848] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:40 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:40.352515 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea19ac0-797d-431b-853d-5cb542ba8727 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:40.353631 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea19ac0-797d-431b-853d-5cb542ba8727 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:40.354152 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea19ac0-797d-431b-853d-5cb542ba8727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:40.408229 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632f26d7-51ab-4585-8b29-250065837eb6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:40.413538 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-8ea19ac0-797d-431b-853d-5cb542ba8727 None None] MFA Rules not processed for user `03a5c87c21744f3c8d06e01ef1839ccb`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:40.414988 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-8ea19ac0-797d-431b-853d-5cb542ba8727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:40.423656 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632f26d7-51ab-4585-8b29-250065837eb6 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:40.438676 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 927/1849] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:31:40 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:31:40.439562 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632f26d7-51ab-4585-8b29-250065837eb6 service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:40.449433 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-632f26d7-51ab-4585-8b29-250065837eb6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:40.466277 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632f26d7-51ab-4585-8b29-250065837eb6 service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:40.473095 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42d0914-6b8d-4e47-aedf-dcb2c1ecc45d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:40.478038 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42d0914-6b8d-4e47-aedf-dcb2c1ecc45d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:40.478448 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42d0914-6b8d-4e47-aedf-dcb2c1ecc45d admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:40.478802 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42d0914-6b8d-4e47-aedf-dcb2c1ecc45d admin admin] PATH_INFO: `/v3/projects/021f79f41d384ac9ba1f6b0756aa580a` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:40.481822 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42d0914-6b8d-4e47-aedf-dcb2c1ecc45d admin admin] RBAC: Authorizing `identity:delete_project(project_id=021f79f41d384ac9ba1f6b0756aa580a)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:40.494914 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42d0914-6b8d-4e47-aedf-dcb2c1ecc45d admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:40.610072 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:40.614242 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 923/1850] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:40 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:40.614242 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea19ac0-797d-431b-853d-5cb542ba8727 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:40.617231 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-8ea19ac0-797d-431b-853d-5cb542ba8727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:40.633130 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b42d0914-6b8d-4e47-aedf-dcb2c1ecc45d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '021f79f41d384ac9ba1f6b0756aa580a', 'request_id': 'req-b42d0914-6b8d-4e47-aedf-dcb2c1ecc45d'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:40.663990 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b42d0914-6b8d-4e47-aedf-dcb2c1ecc45d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '021f79f41d384ac9ba1f6b0756aa580a', 'request_id': 'req-b42d0914-6b8d-4e47-aedf-dcb2c1ecc45d'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:40.689607 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b42d0914-6b8d-4e47-aedf-dcb2c1ecc45d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '021f79f41d384ac9ba1f6b0756aa580a', 'request_id': 'req-b42d0914-6b8d-4e47-aedf-dcb2c1ecc45d'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:40.739699 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b42d0914-6b8d-4e47-aedf-dcb2c1ecc45d admin admin] The token cache is being invalidate because project 021f79f41d384ac9ba1f6b0756aa580a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:31:40.740339 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b42d0914-6b8d-4e47-aedf-dcb2c1ecc45d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:40.742253 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b42d0914-6b8d-4e47-aedf-dcb2c1ecc45d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:40.748809 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 928/1851] 10.222.0.102 () {58 vars in 1238 bytes} [Thu Sep 1 17:31:40 2022] DELETE /identity/v3/projects/021f79f41d384ac9ba1f6b0756aa580a => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:40.750018 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42d0914-6b8d-4e47-aedf-dcb2c1ecc45d admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:40.754118 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-b42d0914-6b8d-4e47-aedf-dcb2c1ecc45d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:40.809642 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea19ac0-797d-431b-853d-5cb542ba8727 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:40.884541 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42d0914-6b8d-4e47-aedf-dcb2c1ecc45d admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:40.912814 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da727161-c09d-4f33-86a5-0e102780c287 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '523ed589872a45a09b6a7675a7fb0b75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:40.913482 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da727161-c09d-4f33-86a5-0e102780c287 service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:40.913771 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da727161-c09d-4f33-86a5-0e102780c287 service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:40.914076 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da727161-c09d-4f33-86a5-0e102780c287 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:40.928247 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-da727161-c09d-4f33-86a5-0e102780c287 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:40.941700 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:40.956351 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f434a808-9fe8-40fc-93f5-d6c70ff0ea06 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '523ed589872a45a09b6a7675a7fb0b75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:40.957103 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f434a808-9fe8-40fc-93f5-d6c70ff0ea06 service cinder] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:40.957226 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f434a808-9fe8-40fc-93f5-d6c70ff0ea06 service cinder] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:40.957509 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f434a808-9fe8-40fc-93f5-d6c70ff0ea06 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:40.962062 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-f434a808-9fe8-40fc-93f5-d6c70ff0ea06 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:41.133726 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f434a808-9fe8-40fc-93f5-d6c70ff0ea06 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:41.135105 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f434a808-9fe8-40fc-93f5-d6c70ff0ea06 service cinder] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:41.147866 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 924/1852] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:31:40 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 535 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:31:41.148761 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f434a808-9fe8-40fc-93f5-d6c70ff0ea06 service cinder] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:41.150399 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-f434a808-9fe8-40fc-93f5-d6c70ff0ea06 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:41.158174 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da727161-c09d-4f33-86a5-0e102780c287 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:41.159391 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da727161-c09d-4f33-86a5-0e102780c287 service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:41.172550 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 929/1853] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:31:40 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:31:41.174122 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da727161-c09d-4f33-86a5-0e102780c287 service cinder] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:41.176403 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-da727161-c09d-4f33-86a5-0e102780c287 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:41.351617 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f434a808-9fe8-40fc-93f5-d6c70ff0ea06 service cinder] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:41.371443 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6eb8c8-9f1f-4654-8cd0-4a408f74b99f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:41.374273 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6eb8c8-9f1f-4654-8cd0-4a408f74b99f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:41.375453 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6eb8c8-9f1f-4654-8cd0-4a408f74b99f admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:41.376102 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6eb8c8-9f1f-4654-8cd0-4a408f74b99f admin admin] PATH_INFO: `/v3/users/9e20cf58fdd841128b677e5d7bd0757e` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:41.378133 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6eb8c8-9f1f-4654-8cd0-4a408f74b99f admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e20cf58fdd841128b677e5d7bd0757e)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:41.381247 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6eb8c8-9f1f-4654-8cd0-4a408f74b99f admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:41.395253 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da727161-c09d-4f33-86a5-0e102780c287 service cinder] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:41.401379 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77c6b61-cdd0-4c99-9fb5-2cb3abda6bf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:41.402296 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77c6b61-cdd0-4c99-9fb5-2cb3abda6bf8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:41.402693 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77c6b61-cdd0-4c99-9fb5-2cb3abda6bf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:41.403010 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77c6b61-cdd0-4c99-9fb5-2cb3abda6bf8 admin admin] PATH_INFO: `/v3/projects/2f1b4194551c4d46bf4efd2d8eebe056` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:41.404818 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77c6b61-cdd0-4c99-9fb5-2cb3abda6bf8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f1b4194551c4d46bf4efd2d8eebe056)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:41.407012 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77c6b61-cdd0-4c99-9fb5-2cb3abda6bf8 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:41.444413 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-cb6eb8c8-9f1f-4654-8cd0-4a408f74b99f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e20cf58fdd841128b677e5d7bd0757e'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:41.445162 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-cb6eb8c8-9f1f-4654-8cd0-4a408f74b99f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e20cf58fdd841128b677e5d7bd0757e'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:41.445967 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:41.464604 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-cb6eb8c8-9f1f-4654-8cd0-4a408f74b99f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e20cf58fdd841128b677e5d7bd0757e'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:41.485016 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-cb6eb8c8-9f1f-4654-8cd0-4a408f74b99f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e20cf58fdd841128b677e5d7bd0757e'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:41.491388 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-cb6eb8c8-9f1f-4654-8cd0-4a408f74b99f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e20cf58fdd841128b677e5d7bd0757e'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:41.496565 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c77c6b61-cdd0-4c99-9fb5-2cb3abda6bf8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f1b4194551c4d46bf4efd2d8eebe056', 'request_id': 'req-c77c6b61-cdd0-4c99-9fb5-2cb3abda6bf8'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:41.518914 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c77c6b61-cdd0-4c99-9fb5-2cb3abda6bf8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f1b4194551c4d46bf4efd2d8eebe056', 'request_id': 'req-c77c6b61-cdd0-4c99-9fb5-2cb3abda6bf8'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:41.532697 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 925/1854] 10.222.0.102 () {58 vars in 1229 bytes} [Thu Sep 1 17:31:41 2022] DELETE /identity/v3/users/9e20cf58fdd841128b677e5d7bd0757e => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:41.534522 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6eb8c8-9f1f-4654-8cd0-4a408f74b99f admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:41.539310 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-cb6eb8c8-9f1f-4654-8cd0-4a408f74b99f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:41.542619 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c77c6b61-cdd0-4c99-9fb5-2cb3abda6bf8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f1b4194551c4d46bf4efd2d8eebe056', 'request_id': 'req-c77c6b61-cdd0-4c99-9fb5-2cb3abda6bf8'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:41.587384 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c77c6b61-cdd0-4c99-9fb5-2cb3abda6bf8 admin admin] The token cache is being invalidate because project 2f1b4194551c4d46bf4efd2d8eebe056 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:31:41.587752 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c77c6b61-cdd0-4c99-9fb5-2cb3abda6bf8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:41.592963 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c77c6b61-cdd0-4c99-9fb5-2cb3abda6bf8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:41.601783 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 930/1855] 10.222.0.102 () {58 vars in 1238 bytes} [Thu Sep 1 17:31:41 2022] DELETE /identity/v3/projects/2f1b4194551c4d46bf4efd2d8eebe056 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:41.602436 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77c6b61-cdd0-4c99-9fb5-2cb3abda6bf8 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:41.603844 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-c77c6b61-cdd0-4c99-9fb5-2cb3abda6bf8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:41.612578 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:41.719936 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6eb8c8-9f1f-4654-8cd0-4a408f74b99f admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:41.736051 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062151f7-d09e-402f-97c7-33ed7900f154 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '523ed589872a45a09b6a7675a7fb0b75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:41.737088 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062151f7-d09e-402f-97c7-33ed7900f154 service cinder] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:41.737873 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062151f7-d09e-402f-97c7-33ed7900f154 service cinder] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:41.738441 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062151f7-d09e-402f-97c7-33ed7900f154 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:41.743169 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-062151f7-d09e-402f-97c7-33ed7900f154 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:41.773556 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77c6b61-cdd0-4c99-9fb5-2cb3abda6bf8 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:41.866483 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062151f7-d09e-402f-97c7-33ed7900f154 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:41.867706 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062151f7-d09e-402f-97c7-33ed7900f154 service cinder] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:41.879725 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 926/1856] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:31:41 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:31:41.881868 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062151f7-d09e-402f-97c7-33ed7900f154 service cinder] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:41.886276 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-062151f7-d09e-402f-97c7-33ed7900f154 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:41.945132 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a56059-e0ef-4f83-b32e-1681aabcbd9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:41.946483 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a56059-e0ef-4f83-b32e-1681aabcbd9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:41.946617 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a56059-e0ef-4f83-b32e-1681aabcbd9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:41.946882 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a56059-e0ef-4f83-b32e-1681aabcbd9f admin admin] PATH_INFO: `/v3/users/8803299d636346c69c74eb1498d6f267` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:41.949089 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a56059-e0ef-4f83-b32e-1681aabcbd9f admin admin] RBAC: Authorizing `identity:delete_user(user_id=8803299d636346c69c74eb1498d6f267)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:41.951992 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a56059-e0ef-4f83-b32e-1681aabcbd9f admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:42.042904 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062151f7-d09e-402f-97c7-33ed7900f154 service cinder] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:42.050751 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968c8f33-b3a9-441c-8e65-d325f18569d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:42.052527 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968c8f33-b3a9-441c-8e65-d325f18569d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:42.053063 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968c8f33-b3a9-441c-8e65-d325f18569d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:42.053747 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968c8f33-b3a9-441c-8e65-d325f18569d1 admin admin] PATH_INFO: `/v3/users/408767787bbd4e8eb07f1af315ab8e45` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:42.055761 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968c8f33-b3a9-441c-8e65-d325f18569d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=408767787bbd4e8eb07f1af315ab8e45)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:42.058854 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968c8f33-b3a9-441c-8e65-d325f18569d1 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:42.096301 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-81a56059-e0ef-4f83-b32e-1681aabcbd9f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8803299d636346c69c74eb1498d6f267'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:42.113898 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:42.126509 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-81a56059-e0ef-4f83-b32e-1681aabcbd9f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8803299d636346c69c74eb1498d6f267'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:42.146320 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-81a56059-e0ef-4f83-b32e-1681aabcbd9f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8803299d636346c69c74eb1498d6f267'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:42.148639 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-81a56059-e0ef-4f83-b32e-1681aabcbd9f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8803299d636346c69c74eb1498d6f267'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:42.149577 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-81a56059-e0ef-4f83-b32e-1681aabcbd9f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8803299d636346c69c74eb1498d6f267'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:42.159211 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-968c8f33-b3a9-441c-8e65-d325f18569d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '408767787bbd4e8eb07f1af315ab8e45'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:42.159915 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-968c8f33-b3a9-441c-8e65-d325f18569d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '408767787bbd4e8eb07f1af315ab8e45'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:42.177265 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-968c8f33-b3a9-441c-8e65-d325f18569d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '408767787bbd4e8eb07f1af315ab8e45'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:42.195573 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-968c8f33-b3a9-441c-8e65-d325f18569d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '408767787bbd4e8eb07f1af315ab8e45'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:42.195934 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 931/1857] 10.222.0.102 () {58 vars in 1229 bytes} [Thu Sep 1 17:31:41 2022] DELETE /identity/v3/users/8803299d636346c69c74eb1498d6f267 => generated 0 bytes in 589 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:42.196553 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-968c8f33-b3a9-441c-8e65-d325f18569d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '408767787bbd4e8eb07f1af315ab8e45'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:42.200239 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942fe66e-ac6b-49e0-9f14-f8066e4385e8 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:42.202352 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942fe66e-ac6b-49e0-9f14-f8066e4385e8 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:42.202523 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942fe66e-ac6b-49e0-9f14-f8066e4385e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:42.202839 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:42.246950 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 927/1858] 10.222.0.102 () {58 vars in 1229 bytes} [Thu Sep 1 17:31:41 2022] DELETE /identity/v3/users/408767787bbd4e8eb07f1af315ab8e45 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:42.252620 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-942fe66e-ac6b-49e0-9f14-f8066e4385e8 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:42.254920 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-942fe66e-ac6b-49e0-9f14-f8066e4385e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:42.408102 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 932/1859] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:42 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:42.416915 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968c8f33-b3a9-441c-8e65-d325f18569d1 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:42.418756 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-968c8f33-b3a9-441c-8e65-d325f18569d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:42.437516 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942fe66e-ac6b-49e0-9f14-f8066e4385e8 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:42.439540 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-942fe66e-ac6b-49e0-9f14-f8066e4385e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:42.449864 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968c8f33-b3a9-441c-8e65-d325f18569d1 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:42.460671 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d1f6c4-1e01-4590-925a-119dbb0b0893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:42.461448 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d1f6c4-1e01-4590-925a-119dbb0b0893 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:42.461766 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d1f6c4-1e01-4590-925a-119dbb0b0893 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:42.462084 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d1f6c4-1e01-4590-925a-119dbb0b0893 admin admin] PATH_INFO: `/v3/projects/0dca75e858bb4bb18bec335728d01390` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:42.463447 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d1f6c4-1e01-4590-925a-119dbb0b0893 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0dca75e858bb4bb18bec335728d01390)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:42.465819 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d1f6c4-1e01-4590-925a-119dbb0b0893 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:42.584262 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-d0d1f6c4-1e01-4590-925a-119dbb0b0893 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0dca75e858bb4bb18bec335728d01390', 'request_id': 'req-d0d1f6c4-1e01-4590-925a-119dbb0b0893'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:42.588051 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942fe66e-ac6b-49e0-9f14-f8066e4385e8 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:42.592411 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-d0d1f6c4-1e01-4590-925a-119dbb0b0893 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0dca75e858bb4bb18bec335728d01390', 'request_id': 'req-d0d1f6c4-1e01-4590-925a-119dbb0b0893'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:42.619456 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-d0d1f6c4-1e01-4590-925a-119dbb0b0893 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0dca75e858bb4bb18bec335728d01390', 'request_id': 'req-d0d1f6c4-1e01-4590-925a-119dbb0b0893'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:42.647509 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-d0d1f6c4-1e01-4590-925a-119dbb0b0893 admin admin] The token cache is being invalidate because project 0dca75e858bb4bb18bec335728d01390 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:31:42.647832 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-d0d1f6c4-1e01-4590-925a-119dbb0b0893 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:42.648551 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-d0d1f6c4-1e01-4590-925a-119dbb0b0893 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:42.650618 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 928/1860] 10.222.0.102 () {58 vars in 1238 bytes} [Thu Sep 1 17:31:42 2022] DELETE /identity/v3/projects/0dca75e858bb4bb18bec335728d01390 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:42.651578 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d1f6c4-1e01-4590-925a-119dbb0b0893 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:42.656743 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-d0d1f6c4-1e01-4590-925a-119dbb0b0893 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:42.695728 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ff7486-310a-4bae-abae-c3a6c48f7d06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:42.696492 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ff7486-310a-4bae-abae-c3a6c48f7d06 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:42.696832 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ff7486-310a-4bae-abae-c3a6c48f7d06 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:42.697600 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ff7486-310a-4bae-abae-c3a6c48f7d06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:42.699933 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-64ff7486-310a-4bae-abae-c3a6c48f7d06 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:42.784419 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:42.829763 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ff7486-310a-4bae-abae-c3a6c48f7d06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:42.832227 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ff7486-310a-4bae-abae-c3a6c48f7d06 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:42.848082 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 933/1861] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:31:42 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 17:31:42.849072 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8addd2e9-3640-43ba-a82c-55c64c1a7a90 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:42.853267 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8addd2e9-3640-43ba-a82c-55c64c1a7a90 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:42.853267 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8addd2e9-3640-43ba-a82c-55c64c1a7a90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:42.886938 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d1f6c4-1e01-4590-925a-119dbb0b0893 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:42.895903 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02583b02-cb60-4a95-aaa7-0bcab99f23f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '523ed589872a45a09b6a7675a7fb0b75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:42.896635 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02583b02-cb60-4a95-aaa7-0bcab99f23f9 service cinder] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:42.897026 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02583b02-cb60-4a95-aaa7-0bcab99f23f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:42.897627 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02583b02-cb60-4a95-aaa7-0bcab99f23f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:42.899916 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-8addd2e9-3640-43ba-a82c-55c64c1a7a90 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:42.902423 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-02583b02-cb60-4a95-aaa7-0bcab99f23f9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:42.902959 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-8addd2e9-3640-43ba-a82c-55c64c1a7a90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:42.922471 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 934/1862] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:42 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:42.923556 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8addd2e9-3640-43ba-a82c-55c64c1a7a90 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:42.926453 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-8addd2e9-3640-43ba-a82c-55c64c1a7a90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:42.939961 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8addd2e9-3640-43ba-a82c-55c64c1a7a90 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:42.945161 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba3bacf-a3fc-475b-9bf0-a6666d06c133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:42.945842 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba3bacf-a3fc-475b-9bf0-a6666d06c133 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:42.946318 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba3bacf-a3fc-475b-9bf0-a6666d06c133 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:42.946644 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba3bacf-a3fc-475b-9bf0-a6666d06c133 admin admin] PATH_INFO: `/v3/projects/1c94345144394c9aba353fbc3b0ba56d` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:42.948338 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba3bacf-a3fc-475b-9bf0-a6666d06c133 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c94345144394c9aba353fbc3b0ba56d)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:42.949990 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba3bacf-a3fc-475b-9bf0-a6666d06c133 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:43.042741 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02583b02-cb60-4a95-aaa7-0bcab99f23f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:43.046737 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02583b02-cb60-4a95-aaa7-0bcab99f23f9 service cinder] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:43.076241 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-2ba3bacf-a3fc-475b-9bf0-a6666d06c133 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c94345144394c9aba353fbc3b0ba56d', 'request_id': 'req-2ba3bacf-a3fc-475b-9bf0-a6666d06c133'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:43.090962 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-2ba3bacf-a3fc-475b-9bf0-a6666d06c133 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c94345144394c9aba353fbc3b0ba56d', 'request_id': 'req-2ba3bacf-a3fc-475b-9bf0-a6666d06c133'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:43.095267 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 929/1863] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:31:42 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:31:43.098272 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808737ca-9bff-4453-8799-0d9ae7cd08c8 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:43.098573 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808737ca-9bff-4453-8799-0d9ae7cd08c8 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:43.098900 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808737ca-9bff-4453-8799-0d9ae7cd08c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:43.118905 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-2ba3bacf-a3fc-475b-9bf0-a6666d06c133 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c94345144394c9aba353fbc3b0ba56d', 'request_id': 'req-2ba3bacf-a3fc-475b-9bf0-a6666d06c133'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:43.127493 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-808737ca-9bff-4453-8799-0d9ae7cd08c8 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:43.129636 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-808737ca-9bff-4453-8799-0d9ae7cd08c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:43.159526 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-2ba3bacf-a3fc-475b-9bf0-a6666d06c133 admin admin] The token cache is being invalidate because project 1c94345144394c9aba353fbc3b0ba56d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:31:43.159807 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-2ba3bacf-a3fc-475b-9bf0-a6666d06c133 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:43.160581 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-2ba3bacf-a3fc-475b-9bf0-a6666d06c133 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:43.162612 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 935/1864] 10.222.0.102 () {58 vars in 1238 bytes} [Thu Sep 1 17:31:42 2022] DELETE /identity/v3/projects/1c94345144394c9aba353fbc3b0ba56d => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:43.167000 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c344e182-55f2-488c-abd8-2d3148301d51 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:43.168255 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c344e182-55f2-488c-abd8-2d3148301d51 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:43.168255 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c344e182-55f2-488c-abd8-2d3148301d51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:43.195643 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:43.207436 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-c344e182-55f2-488c-abd8-2d3148301d51 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:43.209509 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-c344e182-55f2-488c-abd8-2d3148301d51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:43.286568 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 930/1865] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:43 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:43.289562 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808737ca-9bff-4453-8799-0d9ae7cd08c8 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:43.309284 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808737ca-9bff-4453-8799-0d9ae7cd08c8 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:43.337062 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e190f4-33e0-42cb-baf1-c06e908affc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:43.339758 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e190f4-33e0-42cb-baf1-c06e908affc4 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:43.340768 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e190f4-33e0-42cb-baf1-c06e908affc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:43.344333 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e190f4-33e0-42cb-baf1-c06e908affc4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:43.345131 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e190f4-33e0-42cb-baf1-c06e908affc4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:31:43.347324 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e190f4-33e0-42cb-baf1-c06e908affc4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:43.349336 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e190f4-33e0-42cb-baf1-c06e908affc4 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:43.377770 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 936/1866] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:43 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:31:43.384764 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 931/1867] 10.222.0.102 () {58 vars in 1158 bytes} [Thu Sep 1 17:31:43 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:31:43.386111 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c344e182-55f2-488c-abd8-2d3148301d51 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:43.388227 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e65961d-b9a9-424b-9651-201893e4ad9d None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:43.388536 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e65961d-b9a9-424b-9651-201893e4ad9d None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:43.388833 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e65961d-b9a9-424b-9651-201893e4ad9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:43.422246 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c344e182-55f2-488c-abd8-2d3148301d51 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:43.439659 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cafa4ee-11c7-4a08-940b-af552083c3d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:43.441751 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cafa4ee-11c7-4a08-940b-af552083c3d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:43.442683 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cafa4ee-11c7-4a08-940b-af552083c3d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:43.443584 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cafa4ee-11c7-4a08-940b-af552083c3d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:43.445027 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cafa4ee-11c7-4a08-940b-af552083c3d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:43.447943 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cafa4ee-11c7-4a08-940b-af552083c3d1 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:43.460110 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-8e65961d-b9a9-424b-9651-201893e4ad9d None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:43.463502 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-8e65961d-b9a9-424b-9651-201893e4ad9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:43.483832 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 932/1868] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:43 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:43.488316 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e65961d-b9a9-424b-9651-201893e4ad9d None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:43.505070 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e65961d-b9a9-424b-9651-201893e4ad9d None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:43.520425 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533ae291-92a9-452f-9e6b-a7901f6d8470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:43.521144 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533ae291-92a9-452f-9e6b-a7901f6d8470 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:43.521538 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533ae291-92a9-452f-9e6b-a7901f6d8470 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:43.522007 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533ae291-92a9-452f-9e6b-a7901f6d8470 admin admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:43.522855 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533ae291-92a9-452f-9e6b-a7901f6d8470 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:31:43.523165 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533ae291-92a9-452f-9e6b-a7901f6d8470 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:43.524894 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533ae291-92a9-452f-9e6b-a7901f6d8470 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:43.525203 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 937/1869] 10.222.0.102 () {60 vars in 1158 bytes} [Thu Sep 1 17:31:43 2022] POST /identity/v3/projects => generated 366 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:31:43.526259 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cafa4ee-11c7-4a08-940b-af552083c3d1 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:43.540667 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 933/1870] 10.222.0.102 () {58 vars in 1158 bytes} [Thu Sep 1 17:31:43 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:31:43.545745 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533ae291-92a9-452f-9e6b-a7901f6d8470 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:43.559659 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533ae291-92a9-452f-9e6b-a7901f6d8470 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:43.606134 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:43.653647 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cafa4ee-11c7-4a08-940b-af552083c3d1 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:43.659826 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c8b31c-c453-4cd0-8e10-53b3357e362f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:43.660713 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c8b31c-c453-4cd0-8e10-53b3357e362f admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:43.661016 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c8b31c-c453-4cd0-8e10-53b3357e362f admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:43.661322 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c8b31c-c453-4cd0-8e10-53b3357e362f admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:43.662120 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c8b31c-c453-4cd0-8e10-53b3357e362f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:43.664297 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c8b31c-c453-4cd0-8e10-53b3357e362f admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:43.693828 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55ba2df-177d-415d-a645-a6a0c9891ea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:43.694373 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55ba2df-177d-415d-a645-a6a0c9891ea2 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:43.694651 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55ba2df-177d-415d-a645-a6a0c9891ea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:43.694923 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55ba2df-177d-415d-a645-a6a0c9891ea2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:43.695999 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55ba2df-177d-415d-a645-a6a0c9891ea2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:43.697481 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55ba2df-177d-415d-a645-a6a0c9891ea2 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:43.697606 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 938/1871] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:31:43 2022] POST /identity/v3/users => generated 284 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:31:43.702272 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c8b31c-c453-4cd0-8e10-53b3357e362f admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:43.742995 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c8b31c-c453-4cd0-8e10-53b3357e362f admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:43.755042 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b54488-ef38-462e-afbb-8e42ac6bb277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:43.755540 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b54488-ef38-462e-afbb-8e42ac6bb277 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:43.755827 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b54488-ef38-462e-afbb-8e42ac6bb277 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:43.756078 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b54488-ef38-462e-afbb-8e42ac6bb277 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:43.756729 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b54488-ef38-462e-afbb-8e42ac6bb277 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:43.758365 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b54488-ef38-462e-afbb-8e42ac6bb277 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:43.777691 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 934/1872] 10.222.0.102 () {60 vars in 1158 bytes} [Thu Sep 1 17:31:43 2022] POST /identity/v3/projects => generated 394 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:31:43.779972 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55ba2df-177d-415d-a645-a6a0c9891ea2 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:43.792817 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 939/1873] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:31:43 2022] GET /identity/v3/roles => generated 1427 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:31:43.801966 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b54488-ef38-462e-afbb-8e42ac6bb277 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:43.832440 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b54488-ef38-462e-afbb-8e42ac6bb277 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:43.943082 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55ba2df-177d-415d-a645-a6a0c9891ea2 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:43.948558 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88844738-d03d-4c34-a684-6cdebc0f87e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:43.949114 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88844738-d03d-4c34-a684-6cdebc0f87e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:43.949438 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88844738-d03d-4c34-a684-6cdebc0f87e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:43.949711 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88844738-d03d-4c34-a684-6cdebc0f87e5 admin admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:43.950417 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88844738-d03d-4c34-a684-6cdebc0f87e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:43.952862 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88844738-d03d-4c34-a684-6cdebc0f87e5 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:43.963449 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33f9485-1167-45cf-905b-b09ccace46a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:43.964086 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33f9485-1167-45cf-905b-b09ccace46a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:43.964728 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33f9485-1167-45cf-905b-b09ccace46a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:43.964978 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33f9485-1167-45cf-905b-b09ccace46a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:43.965779 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33f9485-1167-45cf-905b-b09ccace46a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:43.967935 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33f9485-1167-45cf-905b-b09ccace46a4 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:43.974661 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 935/1874] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:31:43 2022] POST /identity/v3/users => generated 298 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:31:43.975659 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88844738-d03d-4c34-a684-6cdebc0f87e5 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:43.977322 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-88844738-d03d-4c34-a684-6cdebc0f87e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:43.978263 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 940/1875] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:31:43 2022] GET /identity/v3/roles => generated 1427 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:31:43.979075 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33f9485-1167-45cf-905b-b09ccace46a4 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:43.989351 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33f9485-1167-45cf-905b-b09ccace46a4 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:43.991077 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88844738-d03d-4c34-a684-6cdebc0f87e5 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:43.994637 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3430c084-f92b-47b9-9abc-cb4b0cdb0216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:43.995254 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3430c084-f92b-47b9-9abc-cb4b0cdb0216 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:43.995621 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3430c084-f92b-47b9-9abc-cb4b0cdb0216 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:43.995863 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3430c084-f92b-47b9-9abc-cb4b0cdb0216 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:43.996559 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3430c084-f92b-47b9-9abc-cb4b0cdb0216 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:43.996676 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8cf107-dd3c-4dcc-811c-e0f866b2b3b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:43.997699 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8cf107-dd3c-4dcc-811c-e0f866b2b3b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:43.998097 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8cf107-dd3c-4dcc-811c-e0f866b2b3b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:43.998336 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3430c084-f92b-47b9-9abc-cb4b0cdb0216 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:43.998617 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8cf107-dd3c-4dcc-811c-e0f866b2b3b3 admin admin] PATH_INFO: `/v3/projects/322855688b6f482495822cf7b84460b2` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:44.000152 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8cf107-dd3c-4dcc-811c-e0f866b2b3b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=322855688b6f482495822cf7b84460b2)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:44.002619 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8cf107-dd3c-4dcc-811c-e0f866b2b3b3 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:44.013589 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 941/1876] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:31:43 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:31:44.013589 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3430c084-f92b-47b9-9abc-cb4b0cdb0216 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:44.020155 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:44.025269 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3430c084-f92b-47b9-9abc-cb4b0cdb0216 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:44.031052 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bd6bbb-d537-478c-9c48-c184deb83ac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:44.031750 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bd6bbb-d537-478c-9c48-c184deb83ac7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:44.032010 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bd6bbb-d537-478c-9c48-c184deb83ac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:44.032318 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bd6bbb-d537-478c-9c48-c184deb83ac7 admin admin] PATH_INFO: `/v3/projects/7d7fbb7bab9b422fa8a42b6fabdb080f/users/92b2372b87414b399927e0ac53d269bb/roles/b75cb360925e4ba5a65f14965c103ffd` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:44.053061 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bd6bbb-d537-478c-9c48-c184deb83ac7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d7fbb7bab9b422fa8a42b6fabdb080f, user_id=92b2372b87414b399927e0ac53d269bb, role_id=b75cb360925e4ba5a65f14965c103ffd)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:44.057936 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bd6bbb-d537-478c-9c48-c184deb83ac7 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:44.085103 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 942/1877] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:31:44 2022] PUT /identity/v3/projects/7d7fbb7bab9b422fa8a42b6fabdb080f/users/92b2372b87414b399927e0ac53d269bb/roles/b75cb360925e4ba5a65f14965c103ffd => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:44.095015 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bd6bbb-d537-478c-9c48-c184deb83ac7 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:44.095125 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-6b8cf107-dd3c-4dcc-811c-e0f866b2b3b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '322855688b6f482495822cf7b84460b2', 'request_id': 'req-6b8cf107-dd3c-4dcc-811c-e0f866b2b3b3'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:44.099924 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-6b8cf107-dd3c-4dcc-811c-e0f866b2b3b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '322855688b6f482495822cf7b84460b2', 'request_id': 'req-6b8cf107-dd3c-4dcc-811c-e0f866b2b3b3'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:44.107529 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-6b8cf107-dd3c-4dcc-811c-e0f866b2b3b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '322855688b6f482495822cf7b84460b2', 'request_id': 'req-6b8cf107-dd3c-4dcc-811c-e0f866b2b3b3'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:44.119700 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-6b8cf107-dd3c-4dcc-811c-e0f866b2b3b3 admin admin] The token cache is being invalidate because project 322855688b6f482495822cf7b84460b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:31:44.119996 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-6b8cf107-dd3c-4dcc-811c-e0f866b2b3b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:44.120651 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-6b8cf107-dd3c-4dcc-811c-e0f866b2b3b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:44.123059 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 936/1878] 10.222.0.102 () {58 vars in 1238 bytes} [Thu Sep 1 17:31:43 2022] DELETE /identity/v3/projects/322855688b6f482495822cf7b84460b2 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:44.124283 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0844c61a-5902-4d09-aafc-7b8079eb32d7 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:44.124567 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0844c61a-5902-4d09-aafc-7b8079eb32d7 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:44.124852 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0844c61a-5902-4d09-aafc-7b8079eb32d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:44.154018 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-0844c61a-5902-4d09-aafc-7b8079eb32d7 None None] MFA Rules not processed for user `92b2372b87414b399927e0ac53d269bb`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:44.155186 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-0844c61a-5902-4d09-aafc-7b8079eb32d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:44.176832 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bd6bbb-d537-478c-9c48-c184deb83ac7 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:44.184010 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c7d0df-bf6e-4247-9dea-488f60bbd037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:44.184642 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c7d0df-bf6e-4247-9dea-488f60bbd037 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:44.185562 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c7d0df-bf6e-4247-9dea-488f60bbd037 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:44.185915 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c7d0df-bf6e-4247-9dea-488f60bbd037 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:44.186646 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c7d0df-bf6e-4247-9dea-488f60bbd037 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:44.188973 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c7d0df-bf6e-4247-9dea-488f60bbd037 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:44.201712 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 943/1879] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:31:44 2022] GET /identity/v3/roles => generated 1427 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:31:44.202540 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c7d0df-bf6e-4247-9dea-488f60bbd037 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:44.204337 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-e5c7d0df-bf6e-4247-9dea-488f60bbd037 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:44.221945 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c7d0df-bf6e-4247-9dea-488f60bbd037 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:44.227026 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96750bef-440e-49be-a756-bb868ef11eb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:44.227672 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96750bef-440e-49be-a756-bb868ef11eb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:44.228102 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96750bef-440e-49be-a756-bb868ef11eb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:44.228653 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96750bef-440e-49be-a756-bb868ef11eb6 admin admin] PATH_INFO: `/v3/users/1ac8bef352c24ea5868d1815d5d5d529` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:44.229861 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96750bef-440e-49be-a756-bb868ef11eb6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ac8bef352c24ea5868d1815d5d5d529)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:44.231823 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96750bef-440e-49be-a756-bb868ef11eb6 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:44.240698 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:44.262884 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 937/1880] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:44 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:44.264719 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0844c61a-5902-4d09-aafc-7b8079eb32d7 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:44.266352 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-0844c61a-5902-4d09-aafc-7b8079eb32d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:44.284653 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0844c61a-5902-4d09-aafc-7b8079eb32d7 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:44.289440 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2497eb7f-14a9-4507-9b3a-f14f953ca793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:44.290106 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2497eb7f-14a9-4507-9b3a-f14f953ca793 admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:44.290385 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2497eb7f-14a9-4507-9b3a-f14f953ca793 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:44.290689 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2497eb7f-14a9-4507-9b3a-f14f953ca793 admin admin] PATH_INFO: `/v3/projects/1676e229a4aa402cb91885b15b53fbf0/users/508ed6461139406fab3f5d6927030a54/roles/b75cb360925e4ba5a65f14965c103ffd` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:44.297111 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-96750bef-440e-49be-a756-bb868ef11eb6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ac8bef352c24ea5868d1815d5d5d529'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:44.310677 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2497eb7f-14a9-4507-9b3a-f14f953ca793 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1676e229a4aa402cb91885b15b53fbf0, user_id=508ed6461139406fab3f5d6927030a54, role_id=b75cb360925e4ba5a65f14965c103ffd)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:44.313716 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2497eb7f-14a9-4507-9b3a-f14f953ca793 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:44.329984 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-96750bef-440e-49be-a756-bb868ef11eb6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ac8bef352c24ea5868d1815d5d5d529'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:44.345760 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 938/1881] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:31:44 2022] PUT /identity/v3/projects/1676e229a4aa402cb91885b15b53fbf0/users/508ed6461139406fab3f5d6927030a54/roles/b75cb360925e4ba5a65f14965c103ffd => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:44.347084 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-96750bef-440e-49be-a756-bb868ef11eb6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ac8bef352c24ea5868d1815d5d5d529'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:44.347646 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2497eb7f-14a9-4507-9b3a-f14f953ca793 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:44.348338 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-96750bef-440e-49be-a756-bb868ef11eb6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ac8bef352c24ea5868d1815d5d5d529'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:44.349357 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-96750bef-440e-49be-a756-bb868ef11eb6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ac8bef352c24ea5868d1815d5d5d529'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:44.351722 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-2497eb7f-14a9-4507-9b3a-f14f953ca793 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:44.379235 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 944/1882] 10.222.0.102 () {58 vars in 1229 bytes} [Thu Sep 1 17:31:44 2022] DELETE /identity/v3/users/1ac8bef352c24ea5868d1815d5d5d529 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:44.380673 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a133c98-65e9-42f4-91e6-fe66362daeac None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:44.380956 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a133c98-65e9-42f4-91e6-fe66362daeac None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:44.381234 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a133c98-65e9-42f4-91e6-fe66362daeac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:44.427250 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-7a133c98-65e9-42f4-91e6-fe66362daeac None None] MFA Rules not processed for user `508ed6461139406fab3f5d6927030a54`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:44.429064 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-7a133c98-65e9-42f4-91e6-fe66362daeac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:44.458444 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2497eb7f-14a9-4507-9b3a-f14f953ca793 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:44.480104 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:44.541757 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74d726a-36fd-4f23-b53f-64ec86455946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:44.543118 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74d726a-36fd-4f23-b53f-64ec86455946 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:44.543630 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74d726a-36fd-4f23-b53f-64ec86455946 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:44.544143 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74d726a-36fd-4f23-b53f-64ec86455946 admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:44.545904 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74d726a-36fd-4f23-b53f-64ec86455946 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:44.548457 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74d726a-36fd-4f23-b53f-64ec86455946 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:44.594037 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 945/1883] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:44 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:44.599465 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a133c98-65e9-42f4-91e6-fe66362daeac None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:44.601834 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-7a133c98-65e9-42f4-91e6-fe66362daeac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:44.613024 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 939/1884] 10.222.0.102 () {60 vars in 1158 bytes} [Thu Sep 1 17:31:44 2022] POST /identity/v3/projects => generated 362 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:31:44.614910 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74d726a-36fd-4f23-b53f-64ec86455946 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:44.637878 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a133c98-65e9-42f4-91e6-fe66362daeac None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:44.724379 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8fbcef-9260-43e8-8c22-2e7bde504fcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:44.725054 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8fbcef-9260-43e8-8c22-2e7bde504fcf admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:44.725364 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8fbcef-9260-43e8-8c22-2e7bde504fcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:44.725656 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8fbcef-9260-43e8-8c22-2e7bde504fcf admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:44.726299 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8fbcef-9260-43e8-8c22-2e7bde504fcf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:44.728219 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8fbcef-9260-43e8-8c22-2e7bde504fcf admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:44.751132 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74d726a-36fd-4f23-b53f-64ec86455946 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:44.757280 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08ea54f-884a-4bc0-9958-85c1b23718d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:44.757931 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08ea54f-884a-4bc0-9958-85c1b23718d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:44.758354 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08ea54f-884a-4bc0-9958-85c1b23718d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:44.758638 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08ea54f-884a-4bc0-9958-85c1b23718d5 admin admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:44.759809 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08ea54f-884a-4bc0-9958-85c1b23718d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:44.772155 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08ea54f-884a-4bc0-9958-85c1b23718d5 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:44.772743 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:44.786181 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 946/1885] 10.222.0.102 () {60 vars in 1158 bytes} [Thu Sep 1 17:31:44 2022] POST /identity/v3/projects => generated 394 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:31:44.790204 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8fbcef-9260-43e8-8c22-2e7bde504fcf admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:44.825528 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 940/1886] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:31:44 2022] POST /identity/v3/users => generated 288 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:31:44.827529 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08ea54f-884a-4bc0-9958-85c1b23718d5 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:44.840753 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08ea54f-884a-4bc0-9958-85c1b23718d5 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:44.930245 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be92c20c-316a-4ed5-96d3-5ca02d5c50e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:44.930864 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be92c20c-316a-4ed5-96d3-5ca02d5c50e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:44.931144 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be92c20c-316a-4ed5-96d3-5ca02d5c50e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:44.931416 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be92c20c-316a-4ed5-96d3-5ca02d5c50e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:44.932066 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be92c20c-316a-4ed5-96d3-5ca02d5c50e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:44.933607 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8fbcef-9260-43e8-8c22-2e7bde504fcf admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:44.933823 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be92c20c-316a-4ed5-96d3-5ca02d5c50e9 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:44.938620 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60bd34e-cba8-4b92-ac9b-6d1cf8b512fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:44.939493 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60bd34e-cba8-4b92-ac9b-6d1cf8b512fe admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:44.940204 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60bd34e-cba8-4b92-ac9b-6d1cf8b512fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:44.940204 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60bd34e-cba8-4b92-ac9b-6d1cf8b512fe admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:44.940662 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60bd34e-cba8-4b92-ac9b-6d1cf8b512fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:44.942621 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60bd34e-cba8-4b92-ac9b-6d1cf8b512fe admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:44.945349 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 941/1887] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:31:44 2022] GET /identity/v3/roles => generated 1427 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:31:44.948739 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be92c20c-316a-4ed5-96d3-5ca02d5c50e9 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:44.963934 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be92c20c-316a-4ed5-96d3-5ca02d5c50e9 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:44.976197 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 947/1888] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:31:44 2022] POST /identity/v3/users => generated 304 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:31:44.981168 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60bd34e-cba8-4b92-ac9b-6d1cf8b512fe admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:44.991823 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b8a9fe-d36e-47e3-8b76-a1ce3c3e2228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:44.993096 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b8a9fe-d36e-47e3-8b76-a1ce3c3e2228 admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:44.993514 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b8a9fe-d36e-47e3-8b76-a1ce3c3e2228 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:44.994112 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b8a9fe-d36e-47e3-8b76-a1ce3c3e2228 admin admin] PATH_INFO: `/v3/projects/0b7bc6c0be8646198d742b7441fc6273/users/224f92e34cdc4fe4970441cf68e91b49/roles/36577a6f46d642d4b81805ede9872490` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:45.009706 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60bd34e-cba8-4b92-ac9b-6d1cf8b512fe admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:45.024486 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb45dc09-921f-4be1-98d0-4d1f074198a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:45.025888 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b8a9fe-d36e-47e3-8b76-a1ce3c3e2228 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b7bc6c0be8646198d742b7441fc6273, user_id=224f92e34cdc4fe4970441cf68e91b49, role_id=36577a6f46d642d4b81805ede9872490)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:45.026681 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb45dc09-921f-4be1-98d0-4d1f074198a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:45.027117 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb45dc09-921f-4be1-98d0-4d1f074198a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:45.027912 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb45dc09-921f-4be1-98d0-4d1f074198a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:45.030027 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb45dc09-921f-4be1-98d0-4d1f074198a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:45.034036 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b8a9fe-d36e-47e3-8b76-a1ce3c3e2228 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:45.035485 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb45dc09-921f-4be1-98d0-4d1f074198a7 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:45.058340 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:45.072003 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 948/1889] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:31:44 2022] GET /identity/v3/roles => generated 1427 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:31:45.076888 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb45dc09-921f-4be1-98d0-4d1f074198a7 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:45.078216 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 942/1890] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:31:44 2022] PUT /identity/v3/projects/0b7bc6c0be8646198d742b7441fc6273/users/224f92e34cdc4fe4970441cf68e91b49/roles/36577a6f46d642d4b81805ede9872490 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:31:45.081852 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b8a9fe-d36e-47e3-8b76-a1ce3c3e2228 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:45.094855 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb45dc09-921f-4be1-98d0-4d1f074198a7 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:45.217875 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b8a9fe-d36e-47e3-8b76-a1ce3c3e2228 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:45.228749 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cea9b4-a7ee-4ed4-bd40-b9606d060435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:45.229389 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cea9b4-a7ee-4ed4-bd40-b9606d060435 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:45.229660 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cea9b4-a7ee-4ed4-bd40-b9606d060435 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:45.230121 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cea9b4-a7ee-4ed4-bd40-b9606d060435 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:45.230989 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cea9b4-a7ee-4ed4-bd40-b9606d060435 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:45.234175 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cea9b4-a7ee-4ed4-bd40-b9606d060435 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:45.239510 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203e5d58-cf66-44c8-a229-58170b9c8f23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:45.240319 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203e5d58-cf66-44c8-a229-58170b9c8f23 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:45.240603 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203e5d58-cf66-44c8-a229-58170b9c8f23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:45.240878 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203e5d58-cf66-44c8-a229-58170b9c8f23 admin admin] PATH_INFO: `/v3/projects/ecefca9367b948c783c7c44057def533/users/91d0a5a78da74f11954c98cd16540be2/roles/36577a6f46d642d4b81805ede9872490` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:45.250268 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 943/1891] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:31:45 2022] GET /identity/v3/roles => generated 1427 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 17:31:45.250268 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cea9b4-a7ee-4ed4-bd40-b9606d060435 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:45.250514 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-78cea9b4-a7ee-4ed4-bd40-b9606d060435 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:45.265471 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203e5d58-cf66-44c8-a229-58170b9c8f23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ecefca9367b948c783c7c44057def533, user_id=91d0a5a78da74f11954c98cd16540be2, role_id=36577a6f46d642d4b81805ede9872490)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:45.265691 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cea9b4-a7ee-4ed4-bd40-b9606d060435 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:45.271033 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203e5d58-cf66-44c8-a229-58170b9c8f23 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:45.271235 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b9ffbe-89a7-4e88-acf3-b20864d09f61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:45.271839 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b9ffbe-89a7-4e88-acf3-b20864d09f61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:45.272125 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b9ffbe-89a7-4e88-acf3-b20864d09f61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:45.272402 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b9ffbe-89a7-4e88-acf3-b20864d09f61 admin admin] PATH_INFO: `/v3/projects/cf62ae9589e742d6b6974888e8fb175d` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:45.274363 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b9ffbe-89a7-4e88-acf3-b20864d09f61 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf62ae9589e742d6b6974888e8fb175d)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:45.276096 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b9ffbe-89a7-4e88-acf3-b20864d09f61 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:45.290453 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 949/1892] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:31:45 2022] PUT /identity/v3/projects/ecefca9367b948c783c7c44057def533/users/91d0a5a78da74f11954c98cd16540be2/roles/36577a6f46d642d4b81805ede9872490 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:31:45.291838 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203e5d58-cf66-44c8-a229-58170b9c8f23 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:45.358771 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-b1b9ffbe-89a7-4e88-acf3-b20864d09f61 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf62ae9589e742d6b6974888e8fb175d', 'request_id': 'req-b1b9ffbe-89a7-4e88-acf3-b20864d09f61'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:45.377212 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-b1b9ffbe-89a7-4e88-acf3-b20864d09f61 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf62ae9589e742d6b6974888e8fb175d', 'request_id': 'req-b1b9ffbe-89a7-4e88-acf3-b20864d09f61'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:45.387941 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-b1b9ffbe-89a7-4e88-acf3-b20864d09f61 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf62ae9589e742d6b6974888e8fb175d', 'request_id': 'req-b1b9ffbe-89a7-4e88-acf3-b20864d09f61'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:45.398064 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203e5d58-cf66-44c8-a229-58170b9c8f23 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:45.408788 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:45.410487 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-b1b9ffbe-89a7-4e88-acf3-b20864d09f61 admin admin] The token cache is being invalidate because project cf62ae9589e742d6b6974888e8fb175d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:31:45.411262 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-b1b9ffbe-89a7-4e88-acf3-b20864d09f61 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:45.411767 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-b1b9ffbe-89a7-4e88-acf3-b20864d09f61 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:45.416767 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 944/1893] 10.222.0.102 () {58 vars in 1238 bytes} [Thu Sep 1 17:31:45 2022] DELETE /identity/v3/projects/cf62ae9589e742d6b6974888e8fb175d => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:45.416767 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b9ffbe-89a7-4e88-acf3-b20864d09f61 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:45.416767 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-b1b9ffbe-89a7-4e88-acf3-b20864d09f61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:45.491014 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d2b4fa-2058-43eb-b9c8-cb17085c9a88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:45.491782 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d2b4fa-2058-43eb-b9c8-cb17085c9a88 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:45.492190 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d2b4fa-2058-43eb-b9c8-cb17085c9a88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:45.492487 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d2b4fa-2058-43eb-b9c8-cb17085c9a88 admin admin] PATH_INFO: `/v3/projects/0b7bc6c0be8646198d742b7441fc6273/users/224f92e34cdc4fe4970441cf68e91b49/roles/36577a6f46d642d4b81805ede9872490` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:45.494819 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d2b4fa-2058-43eb-b9c8-cb17085c9a88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b7bc6c0be8646198d742b7441fc6273, user_id=224f92e34cdc4fe4970441cf68e91b49, role_id=36577a6f46d642d4b81805ede9872490)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:45.498394 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d2b4fa-2058-43eb-b9c8-cb17085c9a88 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:45.515613 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b9ffbe-89a7-4e88-acf3-b20864d09f61 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:45.529349 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 950/1894] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:31:45 2022] PUT /identity/v3/projects/0b7bc6c0be8646198d742b7441fc6273/users/224f92e34cdc4fe4970441cf68e91b49/roles/36577a6f46d642d4b81805ede9872490 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:45.529349 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bbe1d1-537d-4360-8a62-f43016da712b None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:45.529349 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bbe1d1-537d-4360-8a62-f43016da712b None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:45.529349 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bbe1d1-537d-4360-8a62-f43016da712b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:45.563300 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:45.564715 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-d1bbe1d1-537d-4360-8a62-f43016da712b None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:45.578182 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-d1bbe1d1-537d-4360-8a62-f43016da712b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:45.659999 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2dcc59-81a2-4c6b-97f4-05072a866c83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:45.660799 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2dcc59-81a2-4c6b-97f4-05072a866c83 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:45.661082 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2dcc59-81a2-4c6b-97f4-05072a866c83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:45.661449 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2dcc59-81a2-4c6b-97f4-05072a866c83 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:45.662157 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2dcc59-81a2-4c6b-97f4-05072a866c83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:45.663915 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2dcc59-81a2-4c6b-97f4-05072a866c83 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:45.679240 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 945/1895] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:31:45 2022] GET /identity/v3/roles => generated 1427 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:31:45.679240 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032779ca-25f2-413d-bd3d-25a06cd170f8 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:45.679240 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032779ca-25f2-413d-bd3d-25a06cd170f8 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:45.679240 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032779ca-25f2-413d-bd3d-25a06cd170f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:45.704976 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 951/1896] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:45 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:45.707218 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bbe1d1-537d-4360-8a62-f43016da712b None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:45.717783 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bbe1d1-537d-4360-8a62-f43016da712b None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:45.727280 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-032779ca-25f2-413d-bd3d-25a06cd170f8 None None] MFA Rules not processed for user `224f92e34cdc4fe4970441cf68e91b49`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:45.728739 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-032779ca-25f2-413d-bd3d-25a06cd170f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:45.750267 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446d924e-2b8b-48e7-a49b-a000a85b4738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:45.751103 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446d924e-2b8b-48e7-a49b-a000a85b4738 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:45.751460 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446d924e-2b8b-48e7-a49b-a000a85b4738 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:45.751743 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446d924e-2b8b-48e7-a49b-a000a85b4738 admin admin] PATH_INFO: `/v3/projects/ecefca9367b948c783c7c44057def533/users/91d0a5a78da74f11954c98cd16540be2/roles/36577a6f46d642d4b81805ede9872490` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:45.753918 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446d924e-2b8b-48e7-a49b-a000a85b4738 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ecefca9367b948c783c7c44057def533, user_id=91d0a5a78da74f11954c98cd16540be2, role_id=36577a6f46d642d4b81805ede9872490)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:45.756795 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446d924e-2b8b-48e7-a49b-a000a85b4738 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:45.784187 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 952/1897] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:31:45 2022] PUT /identity/v3/projects/ecefca9367b948c783c7c44057def533/users/91d0a5a78da74f11954c98cd16540be2/roles/36577a6f46d642d4b81805ede9872490 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:45.786782 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599a93cb-facb-4efd-ab17-3189175a00ed None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:45.789477 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599a93cb-facb-4efd-ab17-3189175a00ed None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:45.790004 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599a93cb-facb-4efd-ab17-3189175a00ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:45.825568 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-599a93cb-facb-4efd-ab17-3189175a00ed None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:45.827167 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-599a93cb-facb-4efd-ab17-3189175a00ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:45.884695 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 946/1898] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:45 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:45.898204 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9440c168-5a69-4d35-a073-2416dbafd694 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:45.898564 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9440c168-5a69-4d35-a073-2416dbafd694 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:45.898941 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9440c168-5a69-4d35-a073-2416dbafd694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:45.967361 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-9440c168-5a69-4d35-a073-2416dbafd694 None None] MFA Rules not processed for user `91d0a5a78da74f11954c98cd16540be2`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:45.969696 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-9440c168-5a69-4d35-a073-2416dbafd694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:45.974794 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 953/1899] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:45 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:45.974794 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599a93cb-facb-4efd-ab17-3189175a00ed None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:45.975473 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-599a93cb-facb-4efd-ab17-3189175a00ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:46.080112 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:46.083010 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 947/1900] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:45 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:46.083933 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9440c168-5a69-4d35-a073-2416dbafd694 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:46.118427 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9440c168-5a69-4d35-a073-2416dbafd694 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:46.122841 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599a93cb-facb-4efd-ab17-3189175a00ed None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:46.126767 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6c0582-efc6-4f64-a04e-3d3b24ee2359 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '523ed589872a45a09b6a7675a7fb0b75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:46.127076 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6245c9-306d-425b-b530-ebe5f1d0cdd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:46.127386 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6c0582-efc6-4f64-a04e-3d3b24ee2359 service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:46.127668 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6c0582-efc6-4f64-a04e-3d3b24ee2359 service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:46.127957 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6c0582-efc6-4f64-a04e-3d3b24ee2359 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:46.129638 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6245c9-306d-425b-b530-ebe5f1d0cdd6 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:46.129944 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6245c9-306d-425b-b530-ebe5f1d0cdd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:46.130567 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6245c9-306d-425b-b530-ebe5f1d0cdd6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:46.131587 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6245c9-306d-425b-b530-ebe5f1d0cdd6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:31:46.131885 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6245c9-306d-425b-b530-ebe5f1d0cdd6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:46.134231 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6245c9-306d-425b-b530-ebe5f1d0cdd6 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:46.152491 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 948/1901] 10.222.0.102 () {58 vars in 1158 bytes} [Thu Sep 1 17:31:46 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:31:46.174847 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6245c9-306d-425b-b530-ebe5f1d0cdd6 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:46.177539 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-fe6245c9-306d-425b-b530-ebe5f1d0cdd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:46.194612 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6245c9-306d-425b-b530-ebe5f1d0cdd6 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:46.201799 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb5d86f-6be7-4df7-a820-5227adb6549c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '523ed589872a45a09b6a7675a7fb0b75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:46.203346 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb5d86f-6be7-4df7-a820-5227adb6549c service cinder] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:46.203634 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb5d86f-6be7-4df7-a820-5227adb6549c service cinder] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:46.203914 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb5d86f-6be7-4df7-a820-5227adb6549c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:46.221247 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb5d86f-6be7-4df7-a820-5227adb6549c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:46.223096 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb5d86f-6be7-4df7-a820-5227adb6549c service cinder] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:46.229007 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:46.233982 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6c0582-efc6-4f64-a04e-3d3b24ee2359 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:46.236383 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6c0582-efc6-4f64-a04e-3d3b24ee2359 service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:46.238895 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 949/1902] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:31:46 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:31:46.239907 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb5d86f-6be7-4df7-a820-5227adb6549c service cinder] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:46.249656 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 954/1903] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:31:45 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:31:46.255683 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb5d86f-6be7-4df7-a820-5227adb6549c service cinder] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:46.270582 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc92d5eb-e9c2-41ff-9ca4-bc8000195464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:46.272087 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc92d5eb-e9c2-41ff-9ca4-bc8000195464 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:46.272087 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc92d5eb-e9c2-41ff-9ca4-bc8000195464 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:46.272298 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc92d5eb-e9c2-41ff-9ca4-bc8000195464 admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:46.273101 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc92d5eb-e9c2-41ff-9ca4-bc8000195464 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:46.275143 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc92d5eb-e9c2-41ff-9ca4-bc8000195464 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:46.341473 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 950/1904] 10.222.0.102 () {60 vars in 1158 bytes} [Thu Sep 1 17:31:46 2022] POST /identity/v3/projects => generated 386 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:31:46.342612 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6c0582-efc6-4f64-a04e-3d3b24ee2359 service cinder] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:46.355008 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc92d5eb-e9c2-41ff-9ca4-bc8000195464 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:46.355554 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6c0582-efc6-4f64-a04e-3d3b24ee2359 service cinder] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:46.433670 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3cede4-b9b1-417d-a4ab-3cdc033bf479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:46.434305 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3cede4-b9b1-417d-a4ab-3cdc033bf479 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:46.434476 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc92d5eb-e9c2-41ff-9ca4-bc8000195464 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:46.434605 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3cede4-b9b1-417d-a4ab-3cdc033bf479 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:46.435358 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3cede4-b9b1-417d-a4ab-3cdc033bf479 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:46.436022 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3cede4-b9b1-417d-a4ab-3cdc033bf479 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:46.437987 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3cede4-b9b1-417d-a4ab-3cdc033bf479 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:46.438704 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be013805-1234-4d7a-bb63-0f5d7b8e61a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '523ed589872a45a09b6a7675a7fb0b75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:46.439424 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be013805-1234-4d7a-bb63-0f5d7b8e61a4 service cinder] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:46.439733 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be013805-1234-4d7a-bb63-0f5d7b8e61a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:46.440008 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be013805-1234-4d7a-bb63-0f5d7b8e61a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:46.441784 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-be013805-1234-4d7a-bb63-0f5d7b8e61a4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:46.460351 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 955/1905] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:31:46 2022] POST /identity/v3/users => generated 294 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:31:46.461189 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3cede4-b9b1-417d-a4ab-3cdc033bf479 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:46.470865 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3cede4-b9b1-417d-a4ab-3cdc033bf479 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:46.475009 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0cb18c-51c7-46b0-a57c-acb5754246fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '523ed589872a45a09b6a7675a7fb0b75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:46.475890 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0cb18c-51c7-46b0-a57c-acb5754246fc service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:46.476056 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0cb18c-51c7-46b0-a57c-acb5754246fc service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:46.476338 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0cb18c-51c7-46b0-a57c-acb5754246fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:46.478378 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-7e0cb18c-51c7-46b0-a57c-acb5754246fc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:46.508173 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be013805-1234-4d7a-bb63-0f5d7b8e61a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:46.509822 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be013805-1234-4d7a-bb63-0f5d7b8e61a4 service cinder] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:46.520216 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 951/1906] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:31:46 2022] GET /identity/v3/auth/tokens => generated 3280 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:31:46.521268 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be013805-1234-4d7a-bb63-0f5d7b8e61a4 service cinder] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:46.533121 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be013805-1234-4d7a-bb63-0f5d7b8e61a4 service cinder] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:46.538950 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bc9a25-777a-4448-887a-b6acec7a1203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:46.539781 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bc9a25-777a-4448-887a-b6acec7a1203 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:46.540233 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bc9a25-777a-4448-887a-b6acec7a1203 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:46.540568 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bc9a25-777a-4448-887a-b6acec7a1203 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:46.541277 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bc9a25-777a-4448-887a-b6acec7a1203 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:46.542957 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bc9a25-777a-4448-887a-b6acec7a1203 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:46.556848 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0cb18c-51c7-46b0-a57c-acb5754246fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:46.558621 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 952/1907] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:31:46 2022] GET /identity/v3/roles => generated 1427 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:31:46.559059 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0cb18c-51c7-46b0-a57c-acb5754246fc service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:46.560641 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bc9a25-777a-4448-887a-b6acec7a1203 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:46.570701 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 956/1908] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:31:46 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:31:46.572084 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bc9a25-777a-4448-887a-b6acec7a1203 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:46.577384 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaddd5c-4bff-455d-9b4d-26eaeacb6dee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:46.577992 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaddd5c-4bff-455d-9b4d-26eaeacb6dee admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:46.578273 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaddd5c-4bff-455d-9b4d-26eaeacb6dee admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:46.578552 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaddd5c-4bff-455d-9b4d-26eaeacb6dee admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:46.579193 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcaddd5c-4bff-455d-9b4d-26eaeacb6dee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:46.580943 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcaddd5c-4bff-455d-9b4d-26eaeacb6dee admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:46.593585 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 953/1909] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:31:46 2022] GET /identity/v3/roles => generated 1427 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:31:46.596879 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0cb18c-51c7-46b0-a57c-acb5754246fc service cinder] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:46.607894 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0cb18c-51c7-46b0-a57c-acb5754246fc service cinder] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:46.613096 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fd0d1b-7e86-4a78-8a0a-0dcbb4be4099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:46.613797 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fd0d1b-7e86-4a78-8a0a-0dcbb4be4099 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:46.614083 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fd0d1b-7e86-4a78-8a0a-0dcbb4be4099 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:46.614409 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fd0d1b-7e86-4a78-8a0a-0dcbb4be4099 admin admin] PATH_INFO: `/v3/projects/293dbdb498dc4c058580392f30a01894/users/e8510c02141d4617b6658a2cae6cb71a/roles/b75cb360925e4ba5a65f14965c103ffd` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:46.617624 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:46.634221 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fd0d1b-7e86-4a78-8a0a-0dcbb4be4099 admin admin] RBAC: Authorizing `identity:create_grant(project_id=293dbdb498dc4c058580392f30a01894, user_id=e8510c02141d4617b6658a2cae6cb71a, role_id=b75cb360925e4ba5a65f14965c103ffd)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:46.637462 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fd0d1b-7e86-4a78-8a0a-0dcbb4be4099 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:46.652764 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 957/1910] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:31:46 2022] PUT /identity/v3/projects/293dbdb498dc4c058580392f30a01894/users/e8510c02141d4617b6658a2cae6cb71a/roles/b75cb360925e4ba5a65f14965c103ffd => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:46.663602 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b501cf-89ec-44dc-a5bf-89102ad49038 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:46.663926 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b501cf-89ec-44dc-a5bf-89102ad49038 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:46.664222 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b501cf-89ec-44dc-a5bf-89102ad49038 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:46.696267 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-18b501cf-89ec-44dc-a5bf-89102ad49038 None None] MFA Rules not processed for user `e8510c02141d4617b6658a2cae6cb71a`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:46.698294 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-18b501cf-89ec-44dc-a5bf-89102ad49038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:46.790811 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:46.797354 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 954/1911] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:46 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:46.799507 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fd0d1b-7e86-4a78-8a0a-0dcbb4be4099 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:46.885555 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fd0d1b-7e86-4a78-8a0a-0dcbb4be4099 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:46.895956 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b501cf-89ec-44dc-a5bf-89102ad49038 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:46.896680 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-18b501cf-89ec-44dc-a5bf-89102ad49038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:46.899627 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bbebaf-7d94-412c-9fd2-cdf68ff2bf9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:46.900376 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bbebaf-7d94-412c-9fd2-cdf68ff2bf9c admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:46.900771 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bbebaf-7d94-412c-9fd2-cdf68ff2bf9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:46.901139 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bbebaf-7d94-412c-9fd2-cdf68ff2bf9c admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:46.902463 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bbebaf-7d94-412c-9fd2-cdf68ff2bf9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:46.905207 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bbebaf-7d94-412c-9fd2-cdf68ff2bf9c admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:46.912290 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b501cf-89ec-44dc-a5bf-89102ad49038 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:46.917115 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c13e157-96af-4c10-b7d9-f1b519b248f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:46.917667 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c13e157-96af-4c10-b7d9-f1b519b248f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:46.917953 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c13e157-96af-4c10-b7d9-f1b519b248f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:46.918235 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c13e157-96af-4c10-b7d9-f1b519b248f6 admin admin] PATH_INFO: `/v3/users/82afc1385f4a4c219ef6de0c2bdfed05` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:46.919946 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c13e157-96af-4c10-b7d9-f1b519b248f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=82afc1385f4a4c219ef6de0c2bdfed05)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:46.921977 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c13e157-96af-4c10-b7d9-f1b519b248f6 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:46.950521 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 958/1912] 10.222.0.102 () {60 vars in 1158 bytes} [Thu Sep 1 17:31:46 2022] POST /identity/v3/projects => generated 388 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:31:46.951557 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bbebaf-7d94-412c-9fd2-cdf68ff2bf9c admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:47.006262 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-1c13e157-96af-4c10-b7d9-f1b519b248f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82afc1385f4a4c219ef6de0c2bdfed05'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:47.007539 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-1c13e157-96af-4c10-b7d9-f1b519b248f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82afc1385f4a4c219ef6de0c2bdfed05'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:47.022252 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bbebaf-7d94-412c-9fd2-cdf68ff2bf9c admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:47.023663 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-1c13e157-96af-4c10-b7d9-f1b519b248f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82afc1385f4a4c219ef6de0c2bdfed05'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:47.027254 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e4fc79-da67-4630-a84d-b06e170b68b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:47.027805 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e4fc79-da67-4630-a84d-b06e170b68b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:47.028083 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e4fc79-da67-4630-a84d-b06e170b68b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:47.028943 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e4fc79-da67-4630-a84d-b06e170b68b0 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:47.029138 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e4fc79-da67-4630-a84d-b06e170b68b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:47.031447 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e4fc79-da67-4630-a84d-b06e170b68b0 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:47.032456 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-1c13e157-96af-4c10-b7d9-f1b519b248f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82afc1385f4a4c219ef6de0c2bdfed05'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:47.033328 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-1c13e157-96af-4c10-b7d9-f1b519b248f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82afc1385f4a4c219ef6de0c2bdfed05'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:47.047713 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 955/1913] 10.222.0.102 () {58 vars in 1229 bytes} [Thu Sep 1 17:31:46 2022] DELETE /identity/v3/users/82afc1385f4a4c219ef6de0c2bdfed05 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:47.053291 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b869b85-f6e0-4011-8bc2-b3e64d71913c None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:47.053809 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b869b85-f6e0-4011-8bc2-b3e64d71913c None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:47.054179 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b869b85-f6e0-4011-8bc2-b3e64d71913c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:47.058414 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 959/1914] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:31:46 2022] POST /identity/v3/users => generated 301 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:31:47.059326 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e4fc79-da67-4630-a84d-b06e170b68b0 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:47.061721 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-41e4fc79-da67-4630-a84d-b06e170b68b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:47.074938 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e4fc79-da67-4630-a84d-b06e170b68b0 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:47.077241 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-4b869b85-f6e0-4011-8bc2-b3e64d71913c None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:47.078461 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-4b869b85-f6e0-4011-8bc2-b3e64d71913c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:47.135517 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe28aa5-15d6-4063-8f4e-aa948ec40854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:47.136175 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe28aa5-15d6-4063-8f4e-aa948ec40854 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:47.136778 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe28aa5-15d6-4063-8f4e-aa948ec40854 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:47.136778 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe28aa5-15d6-4063-8f4e-aa948ec40854 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:47.137394 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 956/1915] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:47 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:47.137523 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe28aa5-15d6-4063-8f4e-aa948ec40854 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:47.139503 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe28aa5-15d6-4063-8f4e-aa948ec40854 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:47.142356 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:47.148700 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b869b85-f6e0-4011-8bc2-b3e64d71913c None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:47.150200 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-4b869b85-f6e0-4011-8bc2-b3e64d71913c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:47.157429 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 960/1916] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:31:47 2022] GET /identity/v3/roles => generated 1427 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 17:31:47.158943 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe28aa5-15d6-4063-8f4e-aa948ec40854 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:47.169156 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe28aa5-15d6-4063-8f4e-aa948ec40854 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:47.186141 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e79bf67-05fd-4605-9cfb-58e63cf7b59d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:47.186926 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e79bf67-05fd-4605-9cfb-58e63cf7b59d admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:47.187288 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e79bf67-05fd-4605-9cfb-58e63cf7b59d admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:47.187651 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e79bf67-05fd-4605-9cfb-58e63cf7b59d admin admin] PATH_INFO: `/v3/projects/f896e79c103f43b6a5ca232f67c5387e/users/0e17f259f0ce487893c36f3bf5b17aa0/roles/36577a6f46d642d4b81805ede9872490` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:47.205941 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e79bf67-05fd-4605-9cfb-58e63cf7b59d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f896e79c103f43b6a5ca232f67c5387e, user_id=0e17f259f0ce487893c36f3bf5b17aa0, role_id=36577a6f46d642d4b81805ede9872490)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:47.209069 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e79bf67-05fd-4605-9cfb-58e63cf7b59d admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:47.214252 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b869b85-f6e0-4011-8bc2-b3e64d71913c None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:47.219766 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2557fc3-de97-45c2-9361-2ba5bbec5c8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:47.220527 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2557fc3-de97-45c2-9361-2ba5bbec5c8d service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:47.220891 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2557fc3-de97-45c2-9361-2ba5bbec5c8d service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:47.221548 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2557fc3-de97-45c2-9361-2ba5bbec5c8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:47.228118 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 961/1917] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:31:47 2022] PUT /identity/v3/projects/f896e79c103f43b6a5ca232f67c5387e/users/0e17f259f0ce487893c36f3bf5b17aa0/roles/36577a6f46d642d4b81805ede9872490 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:47.232989 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e79bf67-05fd-4605-9cfb-58e63cf7b59d admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:47.233138 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2557fc3-de97-45c2-9361-2ba5bbec5c8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:47.234756 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2557fc3-de97-45c2-9361-2ba5bbec5c8d service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:47.246086 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 957/1918] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:31:47 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 17:31:47.306620 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e79bf67-05fd-4605-9cfb-58e63cf7b59d admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:47.312239 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1bb34f-e197-4444-ba4b-193d612cb806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:47.312727 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1bb34f-e197-4444-ba4b-193d612cb806 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:47.313175 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1bb34f-e197-4444-ba4b-193d612cb806 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:47.313679 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1bb34f-e197-4444-ba4b-193d612cb806 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:47.314539 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1bb34f-e197-4444-ba4b-193d612cb806 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:47.316504 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1bb34f-e197-4444-ba4b-193d612cb806 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:47.333225 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 962/1919] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:31:47 2022] GET /identity/v3/roles => generated 1427 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:31:47.334481 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2557fc3-de97-45c2-9361-2ba5bbec5c8d service neutron] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:47.344706 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2557fc3-de97-45c2-9361-2ba5bbec5c8d service neutron] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:47.349825 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a66526-e02b-4297-8bf1-47240d54270a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:47.350468 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a66526-e02b-4297-8bf1-47240d54270a admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:47.350747 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a66526-e02b-4297-8bf1-47240d54270a admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:47.351015 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a66526-e02b-4297-8bf1-47240d54270a admin admin] PATH_INFO: `/v3/projects/f896e79c103f43b6a5ca232f67c5387e/users/0e17f259f0ce487893c36f3bf5b17aa0/roles/36577a6f46d642d4b81805ede9872490` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:47.352916 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a66526-e02b-4297-8bf1-47240d54270a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f896e79c103f43b6a5ca232f67c5387e, user_id=0e17f259f0ce487893c36f3bf5b17aa0, role_id=36577a6f46d642d4b81805ede9872490)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:47.355615 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a66526-e02b-4297-8bf1-47240d54270a admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:47.367048 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 958/1920] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:31:47 2022] PUT /identity/v3/projects/f896e79c103f43b6a5ca232f67c5387e/users/0e17f259f0ce487893c36f3bf5b17aa0/roles/36577a6f46d642d4b81805ede9872490 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:47.377135 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaf0d15-3f73-467e-965a-8936175f142c None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:47.377456 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaf0d15-3f73-467e-965a-8936175f142c None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:47.377734 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaf0d15-3f73-467e-965a-8936175f142c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:47.410672 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-2aaf0d15-3f73-467e-965a-8936175f142c None None] MFA Rules not processed for user `0e17f259f0ce487893c36f3bf5b17aa0`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:47.411898 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-2aaf0d15-3f73-467e-965a-8936175f142c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:47.498308 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 963/1921] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:47 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:47.509375 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a66526-e02b-4297-8bf1-47240d54270a admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:47.510925 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-79a66526-e02b-4297-8bf1-47240d54270a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:47.518126 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:47.601617 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a66526-e02b-4297-8bf1-47240d54270a admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:47.605544 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e36375-7a06-4bf0-8013-2844cecb535f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '523ed589872a45a09b6a7675a7fb0b75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:47.606167 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e36375-7a06-4bf0-8013-2844cecb535f service cinder] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:47.606430 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e36375-7a06-4bf0-8013-2844cecb535f service cinder] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:47.606730 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e36375-7a06-4bf0-8013-2844cecb535f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:47.617662 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e36375-7a06-4bf0-8013-2844cecb535f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:47.619193 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e36375-7a06-4bf0-8013-2844cecb535f service cinder] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:47.622846 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaf0d15-3f73-467e-965a-8936175f142c None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:47.624756 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-2aaf0d15-3f73-467e-965a-8936175f142c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:47.648299 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 959/1922] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:31:47 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:31:47.703408 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:47.729246 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaf0d15-3f73-467e-965a-8936175f142c None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:47.737075 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06023757-79f4-449b-98d7-1a16355be498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:47.737818 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06023757-79f4-449b-98d7-1a16355be498 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:47.738100 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06023757-79f4-449b-98d7-1a16355be498 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:47.738762 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06023757-79f4-449b-98d7-1a16355be498 admin admin] PATH_INFO: `/v3/projects/57c81b82480a4baa97c0530229ac8e36` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:47.740023 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06023757-79f4-449b-98d7-1a16355be498 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57c81b82480a4baa97c0530229ac8e36)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:47.742946 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06023757-79f4-449b-98d7-1a16355be498 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:47.823698 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-06023757-79f4-449b-98d7-1a16355be498 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57c81b82480a4baa97c0530229ac8e36', 'request_id': 'req-06023757-79f4-449b-98d7-1a16355be498'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:47.832498 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-06023757-79f4-449b-98d7-1a16355be498 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57c81b82480a4baa97c0530229ac8e36', 'request_id': 'req-06023757-79f4-449b-98d7-1a16355be498'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:47.840216 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-06023757-79f4-449b-98d7-1a16355be498 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57c81b82480a4baa97c0530229ac8e36', 'request_id': 'req-06023757-79f4-449b-98d7-1a16355be498'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:47.860052 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-06023757-79f4-449b-98d7-1a16355be498 admin admin] The token cache is being invalidate because project 57c81b82480a4baa97c0530229ac8e36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:31:47.860523 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-06023757-79f4-449b-98d7-1a16355be498 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:47.863057 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-06023757-79f4-449b-98d7-1a16355be498 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:47.864938 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 964/1923] 10.222.0.102 () {58 vars in 1238 bytes} [Thu Sep 1 17:31:47 2022] DELETE /identity/v3/projects/57c81b82480a4baa97c0530229ac8e36 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:31:47.868758 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e36375-7a06-4bf0-8013-2844cecb535f service cinder] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:47.880969 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e36375-7a06-4bf0-8013-2844cecb535f service cinder] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:47.974336 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f526568f-08de-4073-aa6e-e2a3935273c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:47.975013 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f526568f-08de-4073-aa6e-e2a3935273c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:47.975281 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f526568f-08de-4073-aa6e-e2a3935273c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:47.975555 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f526568f-08de-4073-aa6e-e2a3935273c9 admin admin] PATH_INFO: `/v3/users/9558cef7df8f4f478666ae56e0ac6169` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:47.976671 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f526568f-08de-4073-aa6e-e2a3935273c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9558cef7df8f4f478666ae56e0ac6169)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:47.978774 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f526568f-08de-4073-aa6e-e2a3935273c9 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:48.035714 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-f526568f-08de-4073-aa6e-e2a3935273c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9558cef7df8f4f478666ae56e0ac6169'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:48.038481 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-f526568f-08de-4073-aa6e-e2a3935273c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9558cef7df8f4f478666ae56e0ac6169'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:48.053419 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-f526568f-08de-4073-aa6e-e2a3935273c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9558cef7df8f4f478666ae56e0ac6169'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:48.063049 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-f526568f-08de-4073-aa6e-e2a3935273c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9558cef7df8f4f478666ae56e0ac6169'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:48.063718 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-f526568f-08de-4073-aa6e-e2a3935273c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9558cef7df8f4f478666ae56e0ac6169'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:48.080783 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 960/1924] 10.222.0.102 () {58 vars in 1229 bytes} [Thu Sep 1 17:31:47 2022] DELETE /identity/v3/users/9558cef7df8f4f478666ae56e0ac6169 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:48.437012 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06023757-79f4-449b-98d7-1a16355be498 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:48.438756 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-06023757-79f4-449b-98d7-1a16355be498 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:48.451946 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f526568f-08de-4073-aa6e-e2a3935273c9 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:48.452961 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06023757-79f4-449b-98d7-1a16355be498 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:48.453673 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-f526568f-08de-4073-aa6e-e2a3935273c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:48.498913 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:48.532445 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a4b740-9031-4c97-bfcb-75d60d5693c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:48.534429 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a4b740-9031-4c97-bfcb-75d60d5693c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:48.534717 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a4b740-9031-4c97-bfcb-75d60d5693c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:48.535005 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a4b740-9031-4c97-bfcb-75d60d5693c2 admin admin] PATH_INFO: `/v3/users/e8510c02141d4617b6658a2cae6cb71a` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:48.536204 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a4b740-9031-4c97-bfcb-75d60d5693c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8510c02141d4617b6658a2cae6cb71a)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:48.538458 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a4b740-9031-4c97-bfcb-75d60d5693c2 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:48.547317 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f526568f-08de-4073-aa6e-e2a3935273c9 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:48.552153 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15922af8-f53f-441e-8b3f-921a4f014ec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:48.552856 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15922af8-f53f-441e-8b3f-921a4f014ec8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:48.553208 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15922af8-f53f-441e-8b3f-921a4f014ec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:48.553590 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15922af8-f53f-441e-8b3f-921a4f014ec8 admin admin] PATH_INFO: `/v3/projects/086fe57f9151486fbd9313f2ce2560ef` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:48.554835 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15922af8-f53f-441e-8b3f-921a4f014ec8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=086fe57f9151486fbd9313f2ce2560ef)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:48.556805 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15922af8-f53f-441e-8b3f-921a4f014ec8 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:48.592766 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-29a4b740-9031-4c97-bfcb-75d60d5693c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8510c02141d4617b6658a2cae6cb71a'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:48.606576 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-15922af8-f53f-441e-8b3f-921a4f014ec8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '086fe57f9151486fbd9313f2ce2560ef', 'request_id': 'req-15922af8-f53f-441e-8b3f-921a4f014ec8'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:48.607282 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-29a4b740-9031-4c97-bfcb-75d60d5693c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8510c02141d4617b6658a2cae6cb71a'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:48.614202 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-15922af8-f53f-441e-8b3f-921a4f014ec8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '086fe57f9151486fbd9313f2ce2560ef', 'request_id': 'req-15922af8-f53f-441e-8b3f-921a4f014ec8'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:48.615636 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-29a4b740-9031-4c97-bfcb-75d60d5693c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8510c02141d4617b6658a2cae6cb71a'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:48.616339 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-29a4b740-9031-4c97-bfcb-75d60d5693c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8510c02141d4617b6658a2cae6cb71a'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:48.617050 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-29a4b740-9031-4c97-bfcb-75d60d5693c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8510c02141d4617b6658a2cae6cb71a'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:48.622077 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-15922af8-f53f-441e-8b3f-921a4f014ec8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '086fe57f9151486fbd9313f2ce2560ef', 'request_id': 'req-15922af8-f53f-441e-8b3f-921a4f014ec8'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:48.630505 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 965/1925] 10.222.0.102 () {58 vars in 1229 bytes} [Thu Sep 1 17:31:48 2022] DELETE /identity/v3/users/e8510c02141d4617b6658a2cae6cb71a => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:48.633703 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-15922af8-f53f-441e-8b3f-921a4f014ec8 admin admin] The token cache is being invalidate because project 086fe57f9151486fbd9313f2ce2560ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:31:48.633991 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-15922af8-f53f-441e-8b3f-921a4f014ec8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:48.634626 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-15922af8-f53f-441e-8b3f-921a4f014ec8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:48.634730 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfabe85-6195-43b1-a97d-3e21fa0db1f6 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:48.635052 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfabe85-6195-43b1-a97d-3e21fa0db1f6 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:48.635330 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfabe85-6195-43b1-a97d-3e21fa0db1f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:48.636074 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 961/1926] 10.222.0.102 () {58 vars in 1238 bytes} [Thu Sep 1 17:31:48 2022] DELETE /identity/v3/projects/086fe57f9151486fbd9313f2ce2560ef => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:31:48.639752 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15922af8-f53f-441e-8b3f-921a4f014ec8 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:48.641024 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-15922af8-f53f-441e-8b3f-921a4f014ec8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:48.642550 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:48.657345 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-3cfabe85-6195-43b1-a97d-3e21fa0db1f6 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:48.659382 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-3cfabe85-6195-43b1-a97d-3e21fa0db1f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:48.714789 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15922af8-f53f-441e-8b3f-921a4f014ec8 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:48.725573 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 966/1927] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:48 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:31:48.735455 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfabe85-6195-43b1-a97d-3e21fa0db1f6 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:48.737446 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-3cfabe85-6195-43b1-a97d-3e21fa0db1f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:48.785647 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b5c72b-4f53-4c6a-9d31-2d49cfe59b79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:48.786385 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b5c72b-4f53-4c6a-9d31-2d49cfe59b79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:48.786942 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b5c72b-4f53-4c6a-9d31-2d49cfe59b79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:48.787532 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b5c72b-4f53-4c6a-9d31-2d49cfe59b79 admin admin] PATH_INFO: `/v3/users/03a5c87c21744f3c8d06e01ef1839ccb` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:48.790029 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b5c72b-4f53-4c6a-9d31-2d49cfe59b79 admin admin] RBAC: Authorizing `identity:delete_user(user_id=03a5c87c21744f3c8d06e01ef1839ccb)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:48.792838 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b5c72b-4f53-4c6a-9d31-2d49cfe59b79 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:48.797054 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:48.825496 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfabe85-6195-43b1-a97d-3e21fa0db1f6 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:48.828737 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c0e472-dcac-4516-aadb-ab2bd9944da4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:48.829466 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c0e472-dcac-4516-aadb-ab2bd9944da4 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:48.829857 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c0e472-dcac-4516-aadb-ab2bd9944da4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:48.830139 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c0e472-dcac-4516-aadb-ab2bd9944da4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:48.844680 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c0e472-dcac-4516-aadb-ab2bd9944da4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:48.846184 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c0e472-dcac-4516-aadb-ab2bd9944da4 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:48.856569 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 967/1928] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:31:48 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 17:31:48.857459 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-48b5c72b-4f53-4c6a-9d31-2d49cfe59b79 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03a5c87c21744f3c8d06e01ef1839ccb'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:48.858183 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-48b5c72b-4f53-4c6a-9d31-2d49cfe59b79 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03a5c87c21744f3c8d06e01ef1839ccb'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:48.870634 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-48b5c72b-4f53-4c6a-9d31-2d49cfe59b79 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03a5c87c21744f3c8d06e01ef1839ccb'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:48.879249 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-48b5c72b-4f53-4c6a-9d31-2d49cfe59b79 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03a5c87c21744f3c8d06e01ef1839ccb'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:48.880005 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-48b5c72b-4f53-4c6a-9d31-2d49cfe59b79 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03a5c87c21744f3c8d06e01ef1839ccb'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:48.892397 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 962/1929] 10.222.0.102 () {58 vars in 1229 bytes} [Thu Sep 1 17:31:48 2022] DELETE /identity/v3/users/03a5c87c21744f3c8d06e01ef1839ccb => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:49.151115 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c0e472-dcac-4516-aadb-ab2bd9944da4 service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:49.152879 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-a7c0e472-dcac-4516-aadb-ab2bd9944da4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:49.169620 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c0e472-dcac-4516-aadb-ab2bd9944da4 service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:49.206493 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b5c72b-4f53-4c6a-9d31-2d49cfe59b79 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:49.207974 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-48b5c72b-4f53-4c6a-9d31-2d49cfe59b79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:49.264894 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dab69a1-f817-456c-819c-3858ff34dc1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:49.265752 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dab69a1-f817-456c-819c-3858ff34dc1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:49.266049 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dab69a1-f817-456c-819c-3858ff34dc1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:49.266323 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dab69a1-f817-456c-819c-3858ff34dc1f admin admin] PATH_INFO: `/v3/projects/293dbdb498dc4c058580392f30a01894` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:49.267523 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dab69a1-f817-456c-819c-3858ff34dc1f admin admin] RBAC: Authorizing `identity:delete_project(project_id=293dbdb498dc4c058580392f30a01894)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:49.269713 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dab69a1-f817-456c-819c-3858ff34dc1f admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:49.284378 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b5c72b-4f53-4c6a-9d31-2d49cfe59b79 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:49.289385 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72e866b-1943-4588-9370-8388338498f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:49.290120 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72e866b-1943-4588-9370-8388338498f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:49.290403 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72e866b-1943-4588-9370-8388338498f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:49.290789 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72e866b-1943-4588-9370-8388338498f9 admin admin] PATH_INFO: `/v3/projects/ab1ac2c673e845a19f8afa8e5437c47e` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:49.292006 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72e866b-1943-4588-9370-8388338498f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab1ac2c673e845a19f8afa8e5437c47e)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:49.294125 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72e866b-1943-4588-9370-8388338498f9 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:49.324068 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:49.326183 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-0dab69a1-f817-456c-819c-3858ff34dc1f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '293dbdb498dc4c058580392f30a01894', 'request_id': 'req-0dab69a1-f817-456c-819c-3858ff34dc1f'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:49.335423 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-0dab69a1-f817-456c-819c-3858ff34dc1f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '293dbdb498dc4c058580392f30a01894', 'request_id': 'req-0dab69a1-f817-456c-819c-3858ff34dc1f'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:49.342782 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-0dab69a1-f817-456c-819c-3858ff34dc1f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '293dbdb498dc4c058580392f30a01894', 'request_id': 'req-0dab69a1-f817-456c-819c-3858ff34dc1f'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:49.354589 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-0dab69a1-f817-456c-819c-3858ff34dc1f admin admin] The token cache is being invalidate because project 293dbdb498dc4c058580392f30a01894 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:31:49.354825 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-0dab69a1-f817-456c-819c-3858ff34dc1f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:49.355697 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-0dab69a1-f817-456c-819c-3858ff34dc1f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:49.358406 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 968/1930] 10.222.0.102 () {58 vars in 1238 bytes} [Thu Sep 1 17:31:49 2022] DELETE /identity/v3/projects/293dbdb498dc4c058580392f30a01894 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:49.364831 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-a72e866b-1943-4588-9370-8388338498f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab1ac2c673e845a19f8afa8e5437c47e', 'request_id': 'req-a72e866b-1943-4588-9370-8388338498f9'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:49.374184 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dab69a1-f817-456c-819c-3858ff34dc1f admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:49.377000 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-0dab69a1-f817-456c-819c-3858ff34dc1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:49.380833 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-a72e866b-1943-4588-9370-8388338498f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab1ac2c673e845a19f8afa8e5437c47e', 'request_id': 'req-a72e866b-1943-4588-9370-8388338498f9'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:49.392793 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-a72e866b-1943-4588-9370-8388338498f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab1ac2c673e845a19f8afa8e5437c47e', 'request_id': 'req-a72e866b-1943-4588-9370-8388338498f9'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:49.406469 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-a72e866b-1943-4588-9370-8388338498f9 admin admin] The token cache is being invalidate because project ab1ac2c673e845a19f8afa8e5437c47e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:31:49.406935 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-a72e866b-1943-4588-9370-8388338498f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:49.407613 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-a72e866b-1943-4588-9370-8388338498f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:49.409346 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 963/1931] 10.222.0.102 () {58 vars in 1238 bytes} [Thu Sep 1 17:31:49 2022] DELETE /identity/v3/projects/ab1ac2c673e845a19f8afa8e5437c47e => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:49.419394 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee623201-9dac-497d-b572-b140a4171963 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:49.419394 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee623201-9dac-497d-b572-b140a4171963 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:49.419394 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee623201-9dac-497d-b572-b140a4171963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:49.455431 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-ee623201-9dac-497d-b572-b140a4171963 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:49.457742 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-ee623201-9dac-497d-b572-b140a4171963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:49.466456 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dab69a1-f817-456c-819c-3858ff34dc1f admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:49.468864 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:49.552527 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafc3ac6-4a84-47f7-a8f4-ed208f9a482c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:49.553166 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafc3ac6-4a84-47f7-a8f4-ed208f9a482c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:49.553521 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafc3ac6-4a84-47f7-a8f4-ed208f9a482c admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:49.553845 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafc3ac6-4a84-47f7-a8f4-ed208f9a482c admin admin] PATH_INFO: `/v3/users/0e17f259f0ce487893c36f3bf5b17aa0` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:49.565170 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafc3ac6-4a84-47f7-a8f4-ed208f9a482c admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e17f259f0ce487893c36f3bf5b17aa0)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:49.569784 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafc3ac6-4a84-47f7-a8f4-ed208f9a482c admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:49.577004 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 964/1932] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:49 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:31:49.586683 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7e28d9-7f25-4234-bf6a-6c2a111cbe8e None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:49.588382 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7e28d9-7f25-4234-bf6a-6c2a111cbe8e None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:49.588777 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7e28d9-7f25-4234-bf6a-6c2a111cbe8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:49.619512 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-dc7e28d9-7f25-4234-bf6a-6c2a111cbe8e None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:49.620990 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-dc7e28d9-7f25-4234-bf6a-6c2a111cbe8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:49.636760 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 965/1933] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:49 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:49.639110 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7e28d9-7f25-4234-bf6a-6c2a111cbe8e None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:49.640621 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-dc7e28d9-7f25-4234-bf6a-6c2a111cbe8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:49.666604 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-aafc3ac6-4a84-47f7-a8f4-ed208f9a482c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e17f259f0ce487893c36f3bf5b17aa0'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:49.676696 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7e28d9-7f25-4234-bf6a-6c2a111cbe8e None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:49.689662 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee899fa-272b-46a8-b8b8-549bae548d8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:49.690315 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee899fa-272b-46a8-b8b8-549bae548d8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:49.690640 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee899fa-272b-46a8-b8b8-549bae548d8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:49.690912 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee899fa-272b-46a8-b8b8-549bae548d8d admin admin] PATH_INFO: `/v3/users/92b2372b87414b399927e0ac53d269bb` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:49.692808 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee899fa-272b-46a8-b8b8-549bae548d8d admin admin] RBAC: Authorizing `identity:delete_user(user_id=92b2372b87414b399927e0ac53d269bb)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:49.694969 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee899fa-272b-46a8-b8b8-549bae548d8d admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:49.701248 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-aafc3ac6-4a84-47f7-a8f4-ed208f9a482c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e17f259f0ce487893c36f3bf5b17aa0'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:49.724694 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-aafc3ac6-4a84-47f7-a8f4-ed208f9a482c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e17f259f0ce487893c36f3bf5b17aa0'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:49.725706 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-aafc3ac6-4a84-47f7-a8f4-ed208f9a482c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e17f259f0ce487893c36f3bf5b17aa0'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:49.726747 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-aafc3ac6-4a84-47f7-a8f4-ed208f9a482c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e17f259f0ce487893c36f3bf5b17aa0'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:49.757069 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 969/1934] 10.222.0.102 () {58 vars in 1229 bytes} [Thu Sep 1 17:31:49 2022] DELETE /identity/v3/users/0e17f259f0ce487893c36f3bf5b17aa0 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:31:49.760461 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafc3ac6-4a84-47f7-a8f4-ed208f9a482c admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:49.764277 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-aafc3ac6-4a84-47f7-a8f4-ed208f9a482c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:49.777198 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:49.783786 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-1ee899fa-272b-46a8-b8b8-549bae548d8d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92b2372b87414b399927e0ac53d269bb'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:49.784480 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-1ee899fa-272b-46a8-b8b8-549bae548d8d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92b2372b87414b399927e0ac53d269bb'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:49.811007 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-1ee899fa-272b-46a8-b8b8-549bae548d8d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92b2372b87414b399927e0ac53d269bb'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:49.831113 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-1ee899fa-272b-46a8-b8b8-549bae548d8d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92b2372b87414b399927e0ac53d269bb'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:49.832444 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-1ee899fa-272b-46a8-b8b8-549bae548d8d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92b2372b87414b399927e0ac53d269bb'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:49.860581 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 966/1935] 10.222.0.102 () {58 vars in 1229 bytes} [Thu Sep 1 17:31:49 2022] DELETE /identity/v3/users/92b2372b87414b399927e0ac53d269bb => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:49.863354 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cd37f5-6dc3-4407-91a8-706d0715ae45 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:49.863703 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cd37f5-6dc3-4407-91a8-706d0715ae45 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:49.863985 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cd37f5-6dc3-4407-91a8-706d0715ae45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:49.894331 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-a8cd37f5-6dc3-4407-91a8-706d0715ae45 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:49.896482 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-a8cd37f5-6dc3-4407-91a8-706d0715ae45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:49.942099 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafc3ac6-4a84-47f7-a8f4-ed208f9a482c admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:50.038578 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 967/1936] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:49 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:50.077975 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4664530f-ae0e-4fa9-9a34-6ee714c69267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:50.079356 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4664530f-ae0e-4fa9-9a34-6ee714c69267 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:50.079852 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4664530f-ae0e-4fa9-9a34-6ee714c69267 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:50.080232 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4664530f-ae0e-4fa9-9a34-6ee714c69267 admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:50.081093 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4664530f-ae0e-4fa9-9a34-6ee714c69267 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:31:50.081841 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4664530f-ae0e-4fa9-9a34-6ee714c69267 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:50.084022 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4664530f-ae0e-4fa9-9a34-6ee714c69267 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:50.102304 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 970/1937] 10.222.0.102 () {58 vars in 1158 bytes} [Thu Sep 1 17:31:49 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:31:50.107733 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cd37f5-6dc3-4407-91a8-706d0715ae45 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:50.109092 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-a8cd37f5-6dc3-4407-91a8-706d0715ae45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:50.122771 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4664530f-ae0e-4fa9-9a34-6ee714c69267 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:50.125256 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-4664530f-ae0e-4fa9-9a34-6ee714c69267 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:50.130951 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cd37f5-6dc3-4407-91a8-706d0715ae45 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:50.144473 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ea2405-bdfc-4aaa-a6af-0ccbcb210084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:50.146466 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ea2405-bdfc-4aaa-a6af-0ccbcb210084 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:50.147915 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ea2405-bdfc-4aaa-a6af-0ccbcb210084 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:50.148369 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ea2405-bdfc-4aaa-a6af-0ccbcb210084 admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:50.148885 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ea2405-bdfc-4aaa-a6af-0ccbcb210084 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:50.156818 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ea2405-bdfc-4aaa-a6af-0ccbcb210084 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:50.205874 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:50.226093 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 968/1938] 10.222.0.102 () {60 vars in 1158 bytes} [Thu Sep 1 17:31:50 2022] POST /identity/v3/projects => generated 368 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:31:50.228250 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ea2405-bdfc-4aaa-a6af-0ccbcb210084 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:50.228387 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4664530f-ae0e-4fa9-9a34-6ee714c69267 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:50.230173 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-d1ea2405-bdfc-4aaa-a6af-0ccbcb210084 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:50.345719 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9484c9-65fa-4d78-a021-517ca46fe4a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:50.346600 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9484c9-65fa-4d78-a021-517ca46fe4a1 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:50.346883 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9484c9-65fa-4d78-a021-517ca46fe4a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:50.347157 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9484c9-65fa-4d78-a021-517ca46fe4a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:50.365955 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ea2405-bdfc-4aaa-a6af-0ccbcb210084 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:50.377700 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:50.382781 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1beef2-9f95-466d-9281-f620ed11be46 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '523ed589872a45a09b6a7675a7fb0b75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:50.383846 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1beef2-9f95-466d-9281-f620ed11be46 service cinder] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:50.385617 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1beef2-9f95-466d-9281-f620ed11be46 service cinder] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:50.385909 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1beef2-9f95-466d-9281-f620ed11be46 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:50.399089 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-6d1beef2-9f95-466d-9281-f620ed11be46 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:50.482155 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9484c9-65fa-4d78-a021-517ca46fe4a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:50.486839 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9484c9-65fa-4d78-a021-517ca46fe4a1 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:50.506329 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 971/1939] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:31:50 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:31:50.508288 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9484c9-65fa-4d78-a021-517ca46fe4a1 service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:50.511089 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1beef2-9f95-466d-9281-f620ed11be46 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:50.514400 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1beef2-9f95-466d-9281-f620ed11be46 service cinder] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:50.535078 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 969/1940] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:31:50 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:31:50.537054 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9484c9-65fa-4d78-a021-517ca46fe4a1 service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:50.543287 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed9a15f-a42d-47d3-bfb1-bf787ae1232c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:50.543982 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed9a15f-a42d-47d3-bfb1-bf787ae1232c admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:50.544404 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed9a15f-a42d-47d3-bfb1-bf787ae1232c admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:50.544923 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed9a15f-a42d-47d3-bfb1-bf787ae1232c admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:50.545688 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed9a15f-a42d-47d3-bfb1-bf787ae1232c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:50.548729 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed9a15f-a42d-47d3-bfb1-bf787ae1232c admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:50.584523 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 972/1941] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:31:50 2022] POST /identity/v3/users => generated 285 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:31:50.588391 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1beef2-9f95-466d-9281-f620ed11be46 service cinder] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:50.605513 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1beef2-9f95-466d-9281-f620ed11be46 service cinder] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:50.615968 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eed1b47-694b-4d43-bfe9-d855bbbd2846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:50.616598 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eed1b47-694b-4d43-bfe9-d855bbbd2846 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:50.616925 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eed1b47-694b-4d43-bfe9-d855bbbd2846 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:50.617203 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eed1b47-694b-4d43-bfe9-d855bbbd2846 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:50.617898 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eed1b47-694b-4d43-bfe9-d855bbbd2846 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:50.619546 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eed1b47-694b-4d43-bfe9-d855bbbd2846 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:50.637371 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 970/1942] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:31:50 2022] GET /identity/v3/roles => generated 1427 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 17:31:50.639355 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed9a15f-a42d-47d3-bfb1-bf787ae1232c admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:50.655313 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed9a15f-a42d-47d3-bfb1-bf787ae1232c admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:50.664450 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14da4dd0-bc53-433b-8e7f-f4d1f69bc441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:50.665936 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14da4dd0-bc53-433b-8e7f-f4d1f69bc441 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:50.665936 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14da4dd0-bc53-433b-8e7f-f4d1f69bc441 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:50.666917 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14da4dd0-bc53-433b-8e7f-f4d1f69bc441 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:50.667756 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14da4dd0-bc53-433b-8e7f-f4d1f69bc441 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:50.670174 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14da4dd0-bc53-433b-8e7f-f4d1f69bc441 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:50.691206 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 973/1943] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:31:50 2022] GET /identity/v3/roles => generated 1427 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:31:50.695644 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eed1b47-694b-4d43-bfe9-d855bbbd2846 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:50.724391 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eed1b47-694b-4d43-bfe9-d855bbbd2846 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:50.739464 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6536f45-4f7e-48f3-b4c7-7055f90220ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:50.747541 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6536f45-4f7e-48f3-b4c7-7055f90220ba admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:50.748807 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6536f45-4f7e-48f3-b4c7-7055f90220ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:50.749260 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6536f45-4f7e-48f3-b4c7-7055f90220ba admin admin] PATH_INFO: `/v3/projects/de1af01c5acc4bbf928e937725b6e9d8/users/71fff485c290405ab1c987a546e76a1c/roles/b75cb360925e4ba5a65f14965c103ffd` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:50.795116 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6536f45-4f7e-48f3-b4c7-7055f90220ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=de1af01c5acc4bbf928e937725b6e9d8, user_id=71fff485c290405ab1c987a546e76a1c, role_id=b75cb360925e4ba5a65f14965c103ffd)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:50.798193 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6536f45-4f7e-48f3-b4c7-7055f90220ba admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:50.824363 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 971/1944] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:31:50 2022] PUT /identity/v3/projects/de1af01c5acc4bbf928e937725b6e9d8/users/71fff485c290405ab1c987a546e76a1c/roles/b75cb360925e4ba5a65f14965c103ffd => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:50.837564 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56891da3-5f0a-4f19-b5cc-ef99dc3a54f0 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:50.842146 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56891da3-5f0a-4f19-b5cc-ef99dc3a54f0 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:50.842146 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56891da3-5f0a-4f19-b5cc-ef99dc3a54f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:50.892117 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-56891da3-5f0a-4f19-b5cc-ef99dc3a54f0 None None] MFA Rules not processed for user `71fff485c290405ab1c987a546e76a1c`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:50.893811 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-56891da3-5f0a-4f19-b5cc-ef99dc3a54f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:51.023763 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 974/1945] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:50 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:51.026639 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6536f45-4f7e-48f3-b4c7-7055f90220ba admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:51.116043 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6536f45-4f7e-48f3-b4c7-7055f90220ba admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:51.131480 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2dd3a5-6cc9-47b1-9c26-0b6d11f08b0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:51.132149 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2dd3a5-6cc9-47b1-9c26-0b6d11f08b0c admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:51.132387 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2dd3a5-6cc9-47b1-9c26-0b6d11f08b0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:51.132663 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2dd3a5-6cc9-47b1-9c26-0b6d11f08b0c admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:51.133335 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2dd3a5-6cc9-47b1-9c26-0b6d11f08b0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:51.135410 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2dd3a5-6cc9-47b1-9c26-0b6d11f08b0c admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:51.164633 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:51.177349 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 972/1946] 10.222.0.102 () {60 vars in 1158 bytes} [Thu Sep 1 17:31:51 2022] POST /identity/v3/projects => generated 368 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:31:51.179286 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56891da3-5f0a-4f19-b5cc-ef99dc3a54f0 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:51.191247 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56891da3-5f0a-4f19-b5cc-ef99dc3a54f0 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:51.340151 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bc2c11-62e1-47e5-8690-58014d338e84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:51.341405 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bc2c11-62e1-47e5-8690-58014d338e84 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:51.341987 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bc2c11-62e1-47e5-8690-58014d338e84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:51.342739 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bc2c11-62e1-47e5-8690-58014d338e84 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:51.343654 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bc2c11-62e1-47e5-8690-58014d338e84 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:51.350953 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bc2c11-62e1-47e5-8690-58014d338e84 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:51.375631 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2dd3a5-6cc9-47b1-9c26-0b6d11f08b0c admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:51.378563 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-2e2dd3a5-6cc9-47b1-9c26-0b6d11f08b0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:51.385831 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:51.399596 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 975/1947] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:31:51 2022] POST /identity/v3/users => generated 291 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:31:51.403321 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bc2c11-62e1-47e5-8690-58014d338e84 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:51.423354 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2dd3a5-6cc9-47b1-9c26-0b6d11f08b0c admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:51.425422 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bc2c11-62e1-47e5-8690-58014d338e84 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:51.435746 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9ec19f-9732-4d4b-97f9-5951d1933d19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:51.436561 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9ec19f-9732-4d4b-97f9-5951d1933d19 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:51.437261 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9ec19f-9732-4d4b-97f9-5951d1933d19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:51.437995 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9ec19f-9732-4d4b-97f9-5951d1933d19 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:51.438778 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9ec19f-9732-4d4b-97f9-5951d1933d19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:51.441594 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9ec19f-9732-4d4b-97f9-5951d1933d19 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:51.442501 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1b5e7c-827b-44f7-93bd-b2bd6c0ef7d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:51.443419 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1b5e7c-827b-44f7-93bd-b2bd6c0ef7d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:51.443419 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1b5e7c-827b-44f7-93bd-b2bd6c0ef7d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:51.451719 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1b5e7c-827b-44f7-93bd-b2bd6c0ef7d9 admin admin] PATH_INFO: `/v3/projects/f896e79c103f43b6a5ca232f67c5387e` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:51.454868 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1b5e7c-827b-44f7-93bd-b2bd6c0ef7d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f896e79c103f43b6a5ca232f67c5387e)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:51.456891 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1b5e7c-827b-44f7-93bd-b2bd6c0ef7d9 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:51.460640 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 976/1948] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:31:51 2022] GET /identity/v3/roles => generated 1427 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:31:51.465812 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9ec19f-9732-4d4b-97f9-5951d1933d19 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:51.489723 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9ec19f-9732-4d4b-97f9-5951d1933d19 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:51.503866 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36ce9a3-2846-4076-9909-d3ac197ad01c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:51.505998 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36ce9a3-2846-4076-9909-d3ac197ad01c admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:51.508966 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36ce9a3-2846-4076-9909-d3ac197ad01c admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:51.509545 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36ce9a3-2846-4076-9909-d3ac197ad01c admin admin] PATH_INFO: `/v3/projects/b31690f5ad9d4e6c97d66d8f8cddd4d8/users/9b16b9e35cc740febdc39f6561d65cb1/roles/36577a6f46d642d4b81805ede9872490` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:51.569816 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36ce9a3-2846-4076-9909-d3ac197ad01c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b31690f5ad9d4e6c97d66d8f8cddd4d8, user_id=9b16b9e35cc740febdc39f6561d65cb1, role_id=36577a6f46d642d4b81805ede9872490)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:51.573058 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36ce9a3-2846-4076-9909-d3ac197ad01c admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:51.590205 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-8b1b5e7c-827b-44f7-93bd-b2bd6c0ef7d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f896e79c103f43b6a5ca232f67c5387e', 'request_id': 'req-8b1b5e7c-827b-44f7-93bd-b2bd6c0ef7d9'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:51.593103 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 977/1949] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:31:51 2022] PUT /identity/v3/projects/b31690f5ad9d4e6c97d66d8f8cddd4d8/users/9b16b9e35cc740febdc39f6561d65cb1/roles/36577a6f46d642d4b81805ede9872490 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:51.597050 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36ce9a3-2846-4076-9909-d3ac197ad01c admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:51.614458 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-8b1b5e7c-827b-44f7-93bd-b2bd6c0ef7d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f896e79c103f43b6a5ca232f67c5387e', 'request_id': 'req-8b1b5e7c-827b-44f7-93bd-b2bd6c0ef7d9'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:51.637365 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-8b1b5e7c-827b-44f7-93bd-b2bd6c0ef7d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f896e79c103f43b6a5ca232f67c5387e', 'request_id': 'req-8b1b5e7c-827b-44f7-93bd-b2bd6c0ef7d9'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:51.665746 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-8b1b5e7c-827b-44f7-93bd-b2bd6c0ef7d9 admin admin] The token cache is being invalidate because project f896e79c103f43b6a5ca232f67c5387e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:31:51.667328 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-8b1b5e7c-827b-44f7-93bd-b2bd6c0ef7d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:51.668582 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-8b1b5e7c-827b-44f7-93bd-b2bd6c0ef7d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:51.680417 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 973/1950] 10.222.0.102 () {58 vars in 1238 bytes} [Thu Sep 1 17:31:51 2022] DELETE /identity/v3/projects/f896e79c103f43b6a5ca232f67c5387e => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:51.681277 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36ce9a3-2846-4076-9909-d3ac197ad01c admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:51.685889 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8dd065-4839-467d-8dc3-a74201ecca54 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:51.689277 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8dd065-4839-467d-8dc3-a74201ecca54 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:51.690789 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8dd065-4839-467d-8dc3-a74201ecca54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:51.691077 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e906bad-2a09-4469-ac84-c23abf032cd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:51.691733 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e906bad-2a09-4469-ac84-c23abf032cd0 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:51.692072 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e906bad-2a09-4469-ac84-c23abf032cd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:51.692716 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e906bad-2a09-4469-ac84-c23abf032cd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:51.693420 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e906bad-2a09-4469-ac84-c23abf032cd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:51.702828 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e906bad-2a09-4469-ac84-c23abf032cd0 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:51.721419 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 978/1951] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:31:51 2022] GET /identity/v3/roles => generated 1427 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 17:31:51.726550 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e906bad-2a09-4469-ac84-c23abf032cd0 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:51.730581 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-9e906bad-2a09-4469-ac84-c23abf032cd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:51.733953 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-4c8dd065-4839-467d-8dc3-a74201ecca54 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:51.735319 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-4c8dd065-4839-467d-8dc3-a74201ecca54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:51.748947 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e906bad-2a09-4469-ac84-c23abf032cd0 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:51.769735 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9868ca0a-e6da-4b49-93d2-1faa7e62ed53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:51.770511 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9868ca0a-e6da-4b49-93d2-1faa7e62ed53 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:51.770914 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9868ca0a-e6da-4b49-93d2-1faa7e62ed53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:51.771242 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9868ca0a-e6da-4b49-93d2-1faa7e62ed53 admin admin] PATH_INFO: `/v3/projects/b31690f5ad9d4e6c97d66d8f8cddd4d8/users/9b16b9e35cc740febdc39f6561d65cb1/roles/36577a6f46d642d4b81805ede9872490` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:51.773774 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9868ca0a-e6da-4b49-93d2-1faa7e62ed53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b31690f5ad9d4e6c97d66d8f8cddd4d8, user_id=9b16b9e35cc740febdc39f6561d65cb1, role_id=36577a6f46d642d4b81805ede9872490)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:51.775239 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 974/1952] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:51 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:31:51.779585 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9868ca0a-e6da-4b49-93d2-1faa7e62ed53 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:51.785282 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2d902d-1813-4f4f-bb44-af62ac1f61b5 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:51.786479 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2d902d-1813-4f4f-bb44-af62ac1f61b5 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:51.786759 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2d902d-1813-4f4f-bb44-af62ac1f61b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:51.794862 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 979/1953] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:31:51 2022] PUT /identity/v3/projects/b31690f5ad9d4e6c97d66d8f8cddd4d8/users/9b16b9e35cc740febdc39f6561d65cb1/roles/36577a6f46d642d4b81805ede9872490 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:51.807410 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faefa1d-5208-4761-bb67-7402cea79d41 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:51.808907 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faefa1d-5208-4761-bb67-7402cea79d41 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:51.809869 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faefa1d-5208-4761-bb67-7402cea79d41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:51.845099 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-db2d902d-1813-4f4f-bb44-af62ac1f61b5 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:51.846718 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-db2d902d-1813-4f4f-bb44-af62ac1f61b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:51.893632 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-8faefa1d-5208-4761-bb67-7402cea79d41 None None] MFA Rules not processed for user `9b16b9e35cc740febdc39f6561d65cb1`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:51.895826 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-8faefa1d-5208-4761-bb67-7402cea79d41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:51.972613 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 975/1954] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:51 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:51.973802 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2d902d-1813-4f4f-bb44-af62ac1f61b5 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:51.976935 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-db2d902d-1813-4f4f-bb44-af62ac1f61b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:52.011094 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2d902d-1813-4f4f-bb44-af62ac1f61b5 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:52.020237 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d5de15-d52b-4fea-ae22-8d6df69ff6f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:52.021231 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d5de15-d52b-4fea-ae22-8d6df69ff6f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:52.021552 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d5de15-d52b-4fea-ae22-8d6df69ff6f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:52.021867 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d5de15-d52b-4fea-ae22-8d6df69ff6f9 admin admin] PATH_INFO: `/v3/users/508ed6461139406fab3f5d6927030a54` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:52.024032 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d5de15-d52b-4fea-ae22-8d6df69ff6f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=508ed6461139406fab3f5d6927030a54)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:52.026710 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d5de15-d52b-4fea-ae22-8d6df69ff6f9 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:52.058311 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 980/1955] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:51 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:52.058311 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8faefa1d-5208-4761-bb67-7402cea79d41 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:52.084847 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8faefa1d-5208-4761-bb67-7402cea79d41 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:52.094349 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30985e79-f589-45e7-add3-8c4c21ea79a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:52.095350 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30985e79-f589-45e7-add3-8c4c21ea79a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:52.095729 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30985e79-f589-45e7-add3-8c4c21ea79a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:52.096405 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30985e79-f589-45e7-add3-8c4c21ea79a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:52.097207 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30985e79-f589-45e7-add3-8c4c21ea79a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:31:52.099678 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30985e79-f589-45e7-add3-8c4c21ea79a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:52.099678 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30985e79-f589-45e7-add3-8c4c21ea79a0 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:52.117350 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 981/1956] 10.222.0.102 () {58 vars in 1158 bytes} [Thu Sep 1 17:31:52 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:31:52.117350 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30985e79-f589-45e7-add3-8c4c21ea79a0 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:52.121204 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-30985e79-f589-45e7-add3-8c4c21ea79a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:52.130392 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-42d5de15-d52b-4fea-ae22-8d6df69ff6f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '508ed6461139406fab3f5d6927030a54'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:52.131358 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-42d5de15-d52b-4fea-ae22-8d6df69ff6f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '508ed6461139406fab3f5d6927030a54'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:52.158098 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-42d5de15-d52b-4fea-ae22-8d6df69ff6f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '508ed6461139406fab3f5d6927030a54'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:52.191326 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-42d5de15-d52b-4fea-ae22-8d6df69ff6f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '508ed6461139406fab3f5d6927030a54'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:52.202496 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-42d5de15-d52b-4fea-ae22-8d6df69ff6f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '508ed6461139406fab3f5d6927030a54'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:52.203177 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:52.204610 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:52.236449 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 976/1957] 10.222.0.102 () {58 vars in 1229 bytes} [Thu Sep 1 17:31:51 2022] DELETE /identity/v3/users/508ed6461139406fab3f5d6927030a54 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:52.237690 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d5de15-d52b-4fea-ae22-8d6df69ff6f9 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:52.240425 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-42d5de15-d52b-4fea-ae22-8d6df69ff6f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:52.257808 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30985e79-f589-45e7-add3-8c4c21ea79a0 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:52.344985 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb0a8ff-bb9e-40ec-add4-8661222ebe28 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '523ed589872a45a09b6a7675a7fb0b75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:52.345969 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb0a8ff-bb9e-40ec-add4-8661222ebe28 service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:52.346979 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb0a8ff-bb9e-40ec-add4-8661222ebe28 service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:52.347354 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb0a8ff-bb9e-40ec-add4-8661222ebe28 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:52.349648 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-edb0a8ff-bb9e-40ec-add4-8661222ebe28 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:52.368574 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d5de15-d52b-4fea-ae22-8d6df69ff6f9 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:52.374535 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4978b314-81a5-4dc3-8028-914e0d8b17af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:52.375236 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4978b314-81a5-4dc3-8028-914e0d8b17af admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:52.375525 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4978b314-81a5-4dc3-8028-914e0d8b17af admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:52.376958 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4978b314-81a5-4dc3-8028-914e0d8b17af admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:52.377656 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4978b314-81a5-4dc3-8028-914e0d8b17af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:52.380011 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4978b314-81a5-4dc3-8028-914e0d8b17af admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:52.431802 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb0a8ff-bb9e-40ec-add4-8661222ebe28 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:52.433174 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb0a8ff-bb9e-40ec-add4-8661222ebe28 service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:52.444273 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 977/1958] 10.222.0.102 () {60 vars in 1158 bytes} [Thu Sep 1 17:31:52 2022] POST /identity/v3/projects => generated 386 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:31:52.445735 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1b4522-a537-46aa-a475-8536ca33b40f None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:52.446147 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 982/1959] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:31:52 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:31:52.446266 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1b4522-a537-46aa-a475-8536ca33b40f None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:52.448232 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3024e126-18a0-4881-8f58-75b4b62bd487 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:52.448518 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3024e126-18a0-4881-8f58-75b4b62bd487 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:52.448789 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3024e126-18a0-4881-8f58-75b4b62bd487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:52.455893 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1b4522-a537-46aa-a475-8536ca33b40f None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:52.460976 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 978/1960] 10.222.0.102 () {54 vars in 926 bytes} [Thu Sep 1 17:31:52 2022] GET /identity => generated 271 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:31:52.461839 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1b4522-a537-46aa-a475-8536ca33b40f None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:52.493225 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-3024e126-18a0-4881-8f58-75b4b62bd487 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:52.496183 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-3024e126-18a0-4881-8f58-75b4b62bd487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:52.582119 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1b4522-a537-46aa-a475-8536ca33b40f None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:52.589987 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c696f5-b57e-4183-8156-6c4b0ecd2d22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:52.591566 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c696f5-b57e-4183-8156-6c4b0ecd2d22 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:52.592082 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c696f5-b57e-4183-8156-6c4b0ecd2d22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:52.592691 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c696f5-b57e-4183-8156-6c4b0ecd2d22 admin admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:52.598324 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c696f5-b57e-4183-8156-6c4b0ecd2d22 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:52.601036 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c696f5-b57e-4183-8156-6c4b0ecd2d22 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:52.626421 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 983/1961] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:52 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:52.628012 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3407c100-3a8c-413b-bf07-8a37010713f3 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:52.628300 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3407c100-3a8c-413b-bf07-8a37010713f3 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:52.628686 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3407c100-3a8c-413b-bf07-8a37010713f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:52.643946 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 979/1962] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:31:52 2022] POST /identity/v3/users => generated 294 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:31:52.645391 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c696f5-b57e-4183-8156-6c4b0ecd2d22 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:52.649176 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-62c696f5-b57e-4183-8156-6c4b0ecd2d22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:52.656501 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-3407c100-3a8c-413b-bf07-8a37010713f3 None None] MFA Rules not processed for user `5f8293316baf4774b62fc11d55d50865`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:52.658198 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-3407c100-3a8c-413b-bf07-8a37010713f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:52.683723 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:52.726121 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c696f5-b57e-4183-8156-6c4b0ecd2d22 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:52.730980 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11893f23-ca14-4eee-8a54-936a699ae03a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '523ed589872a45a09b6a7675a7fb0b75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:52.732968 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11893f23-ca14-4eee-8a54-936a699ae03a service cinder] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:52.733691 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11893f23-ca14-4eee-8a54-936a699ae03a service cinder] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:52.734176 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11893f23-ca14-4eee-8a54-936a699ae03a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:52.736221 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-11893f23-ca14-4eee-8a54-936a699ae03a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:52.738137 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 984/1963] 10.222.0.102 () {58 vars in 1023 bytes} [Thu Sep 1 17:31:52 2022] POST /identity/v3/auth/tokens => generated 3382 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:52.739186 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3407c100-3a8c-413b-bf07-8a37010713f3 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:52.750905 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3407c100-3a8c-413b-bf07-8a37010713f3 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:52.756182 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192f8e4c-b213-4086-b473-d438ba4dad0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:52.756816 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192f8e4c-b213-4086-b473-d438ba4dad0b admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:52.757154 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192f8e4c-b213-4086-b473-d438ba4dad0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:52.757477 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192f8e4c-b213-4086-b473-d438ba4dad0b admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:52.758209 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192f8e4c-b213-4086-b473-d438ba4dad0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:52.759953 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192f8e4c-b213-4086-b473-d438ba4dad0b admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:52.760895 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:52.774731 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 985/1964] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:31:52 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:31:52.774731 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192f8e4c-b213-4086-b473-d438ba4dad0b admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:52.776585 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-192f8e4c-b213-4086-b473-d438ba4dad0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:52.844561 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192f8e4c-b213-4086-b473-d438ba4dad0b admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:52.854822 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a592df1e-d91d-469d-9aef-b950b303f837 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:52.854822 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a592df1e-d91d-469d-9aef-b950b303f837 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:52.857518 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a592df1e-d91d-469d-9aef-b950b303f837 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:52.857518 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a592df1e-d91d-469d-9aef-b950b303f837 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:52.861349 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11893f23-ca14-4eee-8a54-936a699ae03a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:52.863805 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11893f23-ca14-4eee-8a54-936a699ae03a service cinder] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:52.877453 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 980/1965] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:31:52 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:31:52.877879 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a592df1e-d91d-469d-9aef-b950b303f837 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:52.878311 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11893f23-ca14-4eee-8a54-936a699ae03a service cinder] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:52.879973 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-11893f23-ca14-4eee-8a54-936a699ae03a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:52.880863 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a592df1e-d91d-469d-9aef-b950b303f837 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:52.897243 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11893f23-ca14-4eee-8a54-936a699ae03a service cinder] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:52.901868 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 986/1966] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:31:52 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:31:52.902006 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd80a17-96b9-4601-94c1-9ad3aa281d13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:52.902676 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd80a17-96b9-4601-94c1-9ad3aa281d13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:52.902797 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a592df1e-d91d-469d-9aef-b950b303f837 service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:52.903141 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd80a17-96b9-4601-94c1-9ad3aa281d13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:52.903524 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd80a17-96b9-4601-94c1-9ad3aa281d13 admin admin] PATH_INFO: `/v3/projects/7d7fbb7bab9b422fa8a42b6fabdb080f` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:52.905174 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd80a17-96b9-4601-94c1-9ad3aa281d13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d7fbb7bab9b422fa8a42b6fabdb080f)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:52.907597 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd80a17-96b9-4601-94c1-9ad3aa281d13 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:52.913459 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a592df1e-d91d-469d-9aef-b950b303f837 service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:52.922503 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069cfeb5-3b1c-4d05-af89-39d160ae0a42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:52.923198 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069cfeb5-3b1c-4d05-af89-39d160ae0a42 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:52.923552 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069cfeb5-3b1c-4d05-af89-39d160ae0a42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:52.925608 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069cfeb5-3b1c-4d05-af89-39d160ae0a42 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:52.926449 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069cfeb5-3b1c-4d05-af89-39d160ae0a42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:52.928950 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069cfeb5-3b1c-4d05-af89-39d160ae0a42 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:52.944698 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 987/1967] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:31:52 2022] GET /identity/v3/roles => generated 1427 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:31:52.945854 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069cfeb5-3b1c-4d05-af89-39d160ae0a42 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:52.949594 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-069cfeb5-3b1c-4d05-af89-39d160ae0a42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:52.970831 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069cfeb5-3b1c-4d05-af89-39d160ae0a42 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:52.977429 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-2dd80a17-96b9-4601-94c1-9ad3aa281d13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d7fbb7bab9b422fa8a42b6fabdb080f', 'request_id': 'req-2dd80a17-96b9-4601-94c1-9ad3aa281d13'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:52.984740 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-2dd80a17-96b9-4601-94c1-9ad3aa281d13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d7fbb7bab9b422fa8a42b6fabdb080f', 'request_id': 'req-2dd80a17-96b9-4601-94c1-9ad3aa281d13'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:52.995491 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-2dd80a17-96b9-4601-94c1-9ad3aa281d13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d7fbb7bab9b422fa8a42b6fabdb080f', 'request_id': 'req-2dd80a17-96b9-4601-94c1-9ad3aa281d13'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:53.017681 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-2dd80a17-96b9-4601-94c1-9ad3aa281d13 admin admin] The token cache is being invalidate because project 7d7fbb7bab9b422fa8a42b6fabdb080f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:31:53.017983 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-2dd80a17-96b9-4601-94c1-9ad3aa281d13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:53.018721 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-2dd80a17-96b9-4601-94c1-9ad3aa281d13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:53.021704 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 981/1968] 10.222.0.102 () {58 vars in 1238 bytes} [Thu Sep 1 17:31:52 2022] DELETE /identity/v3/projects/7d7fbb7bab9b422fa8a42b6fabdb080f => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:53.022561 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd80a17-96b9-4601-94c1-9ad3aa281d13 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:53.024462 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-2dd80a17-96b9-4601-94c1-9ad3aa281d13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:53.096849 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065cb634-d052-47aa-b37f-cd7b7ad2ffbc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f8293316baf4774b62fc11d55d50865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:53.097430 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065cb634-d052-47aa-b37f-cd7b7ad2ffbc service nova] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:53.097617 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065cb634-d052-47aa-b37f-cd7b7ad2ffbc service nova] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:53.099165 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065cb634-d052-47aa-b37f-cd7b7ad2ffbc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:53.102539 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-065cb634-d052-47aa-b37f-cd7b7ad2ffbc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:53.116301 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065cb634-d052-47aa-b37f-cd7b7ad2ffbc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:53.118237 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065cb634-d052-47aa-b37f-cd7b7ad2ffbc service nova] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:53.133621 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 988/1969] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:31:52 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:31:53.134471 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065cb634-d052-47aa-b37f-cd7b7ad2ffbc service nova] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:53.136094 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-065cb634-d052-47aa-b37f-cd7b7ad2ffbc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:53.173697 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd80a17-96b9-4601-94c1-9ad3aa281d13 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:53.179457 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:53.193119 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d647fddb-bc84-400a-953c-ab1b04d633e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:53.193939 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d647fddb-bc84-400a-953c-ab1b04d633e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:53.194349 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d647fddb-bc84-400a-953c-ab1b04d633e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:53.194581 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d647fddb-bc84-400a-953c-ab1b04d633e5 admin admin] PATH_INFO: `/v3/projects/ebcc9a93cec94f15a80d34cae26f661e/users/5b951d15794a497886b2f65c1e691633/roles/b75cb360925e4ba5a65f14965c103ffd` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:53.260305 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d647fddb-bc84-400a-953c-ab1b04d633e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebcc9a93cec94f15a80d34cae26f661e, user_id=5b951d15794a497886b2f65c1e691633, role_id=b75cb360925e4ba5a65f14965c103ffd)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:53.264183 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d647fddb-bc84-400a-953c-ab1b04d633e5 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:53.286036 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065cb634-d052-47aa-b37f-cd7b7ad2ffbc service nova] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:53.297073 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 982/1970] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:31:53 2022] PUT /identity/v3/projects/ebcc9a93cec94f15a80d34cae26f661e/users/5b951d15794a497886b2f65c1e691633/roles/b75cb360925e4ba5a65f14965c103ffd => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:53.309583 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a31aa1-3bbe-4f74-9180-1949e7c6bc8a None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:53.309939 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a31aa1-3bbe-4f74-9180-1949e7c6bc8a None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:53.310122 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a31aa1-3bbe-4f74-9180-1949e7c6bc8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:53.360859 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-07a31aa1-3bbe-4f74-9180-1949e7c6bc8a None None] MFA Rules not processed for user `5b951d15794a497886b2f65c1e691633`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:53.362543 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-07a31aa1-3bbe-4f74-9180-1949e7c6bc8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:53.387714 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bc2c84-02d2-48f5-ae68-5801a1005ede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:53.388528 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bc2c84-02d2-48f5-ae68-5801a1005ede admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:53.388812 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:53.389258 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bc2c84-02d2-48f5-ae68-5801a1005ede admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:53.389258 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bc2c84-02d2-48f5-ae68-5801a1005ede admin admin] PATH_INFO: `/v3/users/224f92e34cdc4fe4970441cf68e91b49` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:53.390758 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bc2c84-02d2-48f5-ae68-5801a1005ede admin admin] RBAC: Authorizing `identity:delete_user(user_id=224f92e34cdc4fe4970441cf68e91b49)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:53.393471 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bc2c84-02d2-48f5-ae68-5801a1005ede admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:53.475696 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 983/1971] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:53 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:31:53.475696 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a31aa1-3bbe-4f74-9180-1949e7c6bc8a None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:53.475696 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-07a31aa1-3bbe-4f74-9180-1949e7c6bc8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:53.480670 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-89bc2c84-02d2-48f5-ae68-5801a1005ede admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '224f92e34cdc4fe4970441cf68e91b49'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:53.502351 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-89bc2c84-02d2-48f5-ae68-5801a1005ede admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '224f92e34cdc4fe4970441cf68e91b49'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:53.502867 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a31aa1-3bbe-4f74-9180-1949e7c6bc8a None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:53.509012 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f7d64b-3565-4dac-8f9d-11e7cf9178ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:53.509723 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f7d64b-3565-4dac-8f9d-11e7cf9178ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:53.510119 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f7d64b-3565-4dac-8f9d-11e7cf9178ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:53.510313 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f7d64b-3565-4dac-8f9d-11e7cf9178ec admin admin] PATH_INFO: `/v3/projects/1676e229a4aa402cb91885b15b53fbf0` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:53.511437 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f7d64b-3565-4dac-8f9d-11e7cf9178ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=1676e229a4aa402cb91885b15b53fbf0)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:53.515122 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f7d64b-3565-4dac-8f9d-11e7cf9178ec admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:53.516973 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-89bc2c84-02d2-48f5-ae68-5801a1005ede admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '224f92e34cdc4fe4970441cf68e91b49'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:53.517808 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-89bc2c84-02d2-48f5-ae68-5801a1005ede admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '224f92e34cdc4fe4970441cf68e91b49'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:53.518590 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-89bc2c84-02d2-48f5-ae68-5801a1005ede admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '224f92e34cdc4fe4970441cf68e91b49'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:53.541832 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 989/1972] 10.222.0.102 () {58 vars in 1229 bytes} [Thu Sep 1 17:31:53 2022] DELETE /identity/v3/users/224f92e34cdc4fe4970441cf68e91b49 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:53.542652 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bc2c84-02d2-48f5-ae68-5801a1005ede admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:53.544052 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-89bc2c84-02d2-48f5-ae68-5801a1005ede admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:53.599186 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-18f7d64b-3565-4dac-8f9d-11e7cf9178ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1676e229a4aa402cb91885b15b53fbf0', 'request_id': 'req-18f7d64b-3565-4dac-8f9d-11e7cf9178ec'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:53.610417 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-18f7d64b-3565-4dac-8f9d-11e7cf9178ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1676e229a4aa402cb91885b15b53fbf0', 'request_id': 'req-18f7d64b-3565-4dac-8f9d-11e7cf9178ec'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:53.624378 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-18f7d64b-3565-4dac-8f9d-11e7cf9178ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1676e229a4aa402cb91885b15b53fbf0', 'request_id': 'req-18f7d64b-3565-4dac-8f9d-11e7cf9178ec'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:53.643872 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-18f7d64b-3565-4dac-8f9d-11e7cf9178ec admin admin] The token cache is being invalidate because project 1676e229a4aa402cb91885b15b53fbf0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:31:53.644547 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-18f7d64b-3565-4dac-8f9d-11e7cf9178ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:53.645460 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-18f7d64b-3565-4dac-8f9d-11e7cf9178ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:53.645963 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bc2c84-02d2-48f5-ae68-5801a1005ede admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:53.647804 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 984/1973] 10.222.0.102 () {58 vars in 1238 bytes} [Thu Sep 1 17:31:53 2022] DELETE /identity/v3/projects/1676e229a4aa402cb91885b15b53fbf0 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:53.653072 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:53.654764 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f7d64b-3565-4dac-8f9d-11e7cf9178ec admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:53.656671 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-18f7d64b-3565-4dac-8f9d-11e7cf9178ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:53.726997 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42450a3c-7ace-4bf3-b6ba-72810741019a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '523ed589872a45a09b6a7675a7fb0b75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:53.727825 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42450a3c-7ace-4bf3-b6ba-72810741019a service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:53.728188 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42450a3c-7ace-4bf3-b6ba-72810741019a service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:53.728676 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42450a3c-7ace-4bf3-b6ba-72810741019a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:53.730783 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-42450a3c-7ace-4bf3-b6ba-72810741019a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:53.827384 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f7d64b-3565-4dac-8f9d-11e7cf9178ec admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:53.841023 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42450a3c-7ace-4bf3-b6ba-72810741019a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:53.841803 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b9e14d-580d-4993-9593-da593c1bb2d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:53.842552 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b9e14d-580d-4993-9593-da593c1bb2d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:53.842974 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b9e14d-580d-4993-9593-da593c1bb2d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:53.843315 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42450a3c-7ace-4bf3-b6ba-72810741019a service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:53.843424 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b9e14d-580d-4993-9593-da593c1bb2d7 admin admin] PATH_INFO: `/v3/users/91d0a5a78da74f11954c98cd16540be2` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:53.845152 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b9e14d-580d-4993-9593-da593c1bb2d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=91d0a5a78da74f11954c98cd16540be2)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:53.852250 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b9e14d-580d-4993-9593-da593c1bb2d7 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:53.885485 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 990/1974] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:31:53 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:31:53.901048 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42450a3c-7ace-4bf3-b6ba-72810741019a service cinder] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:53.904348 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-42450a3c-7ace-4bf3-b6ba-72810741019a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:53.922800 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:53.930752 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-d8b9e14d-580d-4993-9593-da593c1bb2d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91d0a5a78da74f11954c98cd16540be2'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:53.931704 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-d8b9e14d-580d-4993-9593-da593c1bb2d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91d0a5a78da74f11954c98cd16540be2'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:53.932367 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42450a3c-7ace-4bf3-b6ba-72810741019a service cinder] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:53.941612 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caa1113-7538-41ef-b6fc-0a526b7e96f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:53.942490 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caa1113-7538-41ef-b6fc-0a526b7e96f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:53.943441 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caa1113-7538-41ef-b6fc-0a526b7e96f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:53.944003 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caa1113-7538-41ef-b6fc-0a526b7e96f0 admin admin] PATH_INFO: `/v3/users/0d38c1bdca814dbf8a61184a11074575` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:53.945418 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caa1113-7538-41ef-b6fc-0a526b7e96f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d38c1bdca814dbf8a61184a11074575)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:53.947606 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caa1113-7538-41ef-b6fc-0a526b7e96f0 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:53.957488 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-d8b9e14d-580d-4993-9593-da593c1bb2d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91d0a5a78da74f11954c98cd16540be2'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:53.968885 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-d8b9e14d-580d-4993-9593-da593c1bb2d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91d0a5a78da74f11954c98cd16540be2'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:53.970588 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-d8b9e14d-580d-4993-9593-da593c1bb2d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91d0a5a78da74f11954c98cd16540be2'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:53.996066 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 985/1975] 10.222.0.102 () {58 vars in 1229 bytes} [Thu Sep 1 17:31:53 2022] DELETE /identity/v3/users/91d0a5a78da74f11954c98cd16540be2 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:53.996066 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b9e14d-580d-4993-9593-da593c1bb2d7 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:53.996281 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-d8b9e14d-580d-4993-9593-da593c1bb2d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:54.037671 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-5caa1113-7538-41ef-b6fc-0a526b7e96f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d38c1bdca814dbf8a61184a11074575'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:54.058029 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-5caa1113-7538-41ef-b6fc-0a526b7e96f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d38c1bdca814dbf8a61184a11074575'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:54.071715 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-5caa1113-7538-41ef-b6fc-0a526b7e96f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d38c1bdca814dbf8a61184a11074575'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:54.073595 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-5caa1113-7538-41ef-b6fc-0a526b7e96f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d38c1bdca814dbf8a61184a11074575'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:54.074645 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-5caa1113-7538-41ef-b6fc-0a526b7e96f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d38c1bdca814dbf8a61184a11074575'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:54.100452 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 991/1976] 10.222.0.102 () {58 vars in 1229 bytes} [Thu Sep 1 17:31:53 2022] DELETE /identity/v3/users/0d38c1bdca814dbf8a61184a11074575 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:54.100452 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caa1113-7538-41ef-b6fc-0a526b7e96f0 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:54.101106 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-5caa1113-7538-41ef-b6fc-0a526b7e96f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:54.115636 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b9e14d-580d-4993-9593-da593c1bb2d7 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:54.147052 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:54.190544 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caa1113-7538-41ef-b6fc-0a526b7e96f0 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:54.196080 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1451cbe-5abf-41c8-8ce6-c53efab793b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:54.196239 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c369ca1-e0c8-41d8-8585-64f2323888f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:54.196729 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1451cbe-5abf-41c8-8ce6-c53efab793b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:54.196840 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c369ca1-e0c8-41d8-8585-64f2323888f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:54.197013 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1451cbe-5abf-41c8-8ce6-c53efab793b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:54.197107 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c369ca1-e0c8-41d8-8585-64f2323888f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:54.197286 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1451cbe-5abf-41c8-8ce6-c53efab793b6 admin admin] PATH_INFO: `/v3/users/5b951d15794a497886b2f65c1e691633` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:54.197431 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c369ca1-e0c8-41d8-8585-64f2323888f8 admin admin] PATH_INFO: `/v3/projects/0b7bc6c0be8646198d742b7441fc6273` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:54.198518 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c369ca1-e0c8-41d8-8585-64f2323888f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b7bc6c0be8646198d742b7441fc6273)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:54.198635 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1451cbe-5abf-41c8-8ce6-c53efab793b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b951d15794a497886b2f65c1e691633)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:54.201113 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c369ca1-e0c8-41d8-8585-64f2323888f8 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:54.201241 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1451cbe-5abf-41c8-8ce6-c53efab793b6 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:54.254621 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-c1451cbe-5abf-41c8-8ce6-c53efab793b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b951d15794a497886b2f65c1e691633'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:54.255763 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-c1451cbe-5abf-41c8-8ce6-c53efab793b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b951d15794a497886b2f65c1e691633'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:54.257532 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-4c369ca1-e0c8-41d8-8585-64f2323888f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b7bc6c0be8646198d742b7441fc6273', 'request_id': 'req-4c369ca1-e0c8-41d8-8585-64f2323888f8'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:54.267953 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-4c369ca1-e0c8-41d8-8585-64f2323888f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b7bc6c0be8646198d742b7441fc6273', 'request_id': 'req-4c369ca1-e0c8-41d8-8585-64f2323888f8'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:54.273580 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-c1451cbe-5abf-41c8-8ce6-c53efab793b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b951d15794a497886b2f65c1e691633'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:54.275523 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:54.278252 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-4c369ca1-e0c8-41d8-8585-64f2323888f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b7bc6c0be8646198d742b7441fc6273', 'request_id': 'req-4c369ca1-e0c8-41d8-8585-64f2323888f8'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:54.287352 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-c1451cbe-5abf-41c8-8ce6-c53efab793b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b951d15794a497886b2f65c1e691633'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:54.288691 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-c1451cbe-5abf-41c8-8ce6-c53efab793b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b951d15794a497886b2f65c1e691633'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:54.305208 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-4c369ca1-e0c8-41d8-8585-64f2323888f8 admin admin] The token cache is being invalidate because project 0b7bc6c0be8646198d742b7441fc6273 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:31:54.305699 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-4c369ca1-e0c8-41d8-8585-64f2323888f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:54.307184 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-4c369ca1-e0c8-41d8-8585-64f2323888f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:54.311345 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 992/1977] 10.222.0.102 () {58 vars in 1238 bytes} [Thu Sep 1 17:31:54 2022] DELETE /identity/v3/projects/0b7bc6c0be8646198d742b7441fc6273 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:54.320460 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20255d77-fa4d-4028-9d59-400ff72b32af None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:54.321646 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20255d77-fa4d-4028-9d59-400ff72b32af None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:54.322074 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20255d77-fa4d-4028-9d59-400ff72b32af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:54.322299 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 986/1978] 10.222.0.102 () {58 vars in 1229 bytes} [Thu Sep 1 17:31:53 2022] DELETE /identity/v3/users/5b951d15794a497886b2f65c1e691633 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:54.323264 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1451cbe-5abf-41c8-8ce6-c53efab793b6 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:54.324675 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-c1451cbe-5abf-41c8-8ce6-c53efab793b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:54.360031 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-20255d77-fa4d-4028-9d59-400ff72b32af None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:54.361614 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-20255d77-fa4d-4028-9d59-400ff72b32af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:54.404432 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1451cbe-5abf-41c8-8ce6-c53efab793b6 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:54.410020 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eee5180-bb50-4855-a2eb-73e54fe82f0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:54.410739 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eee5180-bb50-4855-a2eb-73e54fe82f0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:54.411028 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eee5180-bb50-4855-a2eb-73e54fe82f0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:54.411305 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eee5180-bb50-4855-a2eb-73e54fe82f0e admin admin] PATH_INFO: `/v3/projects/ecefca9367b948c783c7c44057def533` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:54.412454 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eee5180-bb50-4855-a2eb-73e54fe82f0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ecefca9367b948c783c7c44057def533)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:54.414423 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eee5180-bb50-4855-a2eb-73e54fe82f0e admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:54.426515 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 993/1979] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:54 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:54.427813 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69223a4a-8431-4197-b34f-1fa19164e97e None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:54.428096 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69223a4a-8431-4197-b34f-1fa19164e97e None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:54.428370 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69223a4a-8431-4197-b34f-1fa19164e97e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:54.450435 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-69223a4a-8431-4197-b34f-1fa19164e97e None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:54.452093 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-69223a4a-8431-4197-b34f-1fa19164e97e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:54.469911 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 994/1980] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:54 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:54.473254 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35abb673-50ad-4910-a336-3b60c5e82c65 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:54.473254 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35abb673-50ad-4910-a336-3b60c5e82c65 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:54.473254 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35abb673-50ad-4910-a336-3b60c5e82c65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:54.492279 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-7eee5180-bb50-4855-a2eb-73e54fe82f0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ecefca9367b948c783c7c44057def533', 'request_id': 'req-7eee5180-bb50-4855-a2eb-73e54fe82f0e'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:54.495631 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-35abb673-50ad-4910-a336-3b60c5e82c65 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:54.496826 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-35abb673-50ad-4910-a336-3b60c5e82c65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:54.498930 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-7eee5180-bb50-4855-a2eb-73e54fe82f0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ecefca9367b948c783c7c44057def533', 'request_id': 'req-7eee5180-bb50-4855-a2eb-73e54fe82f0e'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:54.506975 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-7eee5180-bb50-4855-a2eb-73e54fe82f0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ecefca9367b948c783c7c44057def533', 'request_id': 'req-7eee5180-bb50-4855-a2eb-73e54fe82f0e'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:54.527914 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-7eee5180-bb50-4855-a2eb-73e54fe82f0e admin admin] The token cache is being invalidate because project ecefca9367b948c783c7c44057def533 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:31:54.528265 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-7eee5180-bb50-4855-a2eb-73e54fe82f0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:54.529061 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-7eee5180-bb50-4855-a2eb-73e54fe82f0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:54.530623 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 987/1981] 10.222.0.102 () {58 vars in 1238 bytes} [Thu Sep 1 17:31:54 2022] DELETE /identity/v3/projects/ecefca9367b948c783c7c44057def533 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:54.531662 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eee5180-bb50-4855-a2eb-73e54fe82f0e admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:54.532994 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-7eee5180-bb50-4855-a2eb-73e54fe82f0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:54.560758 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:54.573672 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 995/1982] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:54 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:54.575338 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f21ef1-85fe-4571-b40d-b2b36ac4936e None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:54.575781 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f21ef1-85fe-4571-b40d-b2b36ac4936e None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:54.576160 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f21ef1-85fe-4571-b40d-b2b36ac4936e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:54.599646 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-01f21ef1-85fe-4571-b40d-b2b36ac4936e None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:54.600209 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eee5180-bb50-4855-a2eb-73e54fe82f0e admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:54.601017 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-01f21ef1-85fe-4571-b40d-b2b36ac4936e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:54.603915 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a8178b-5ca8-4e1b-b34b-0e916c6c4e8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:54.604524 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a8178b-5ca8-4e1b-b34b-0e916c6c4e8d service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:54.604804 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a8178b-5ca8-4e1b-b34b-0e916c6c4e8d service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:54.605125 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a8178b-5ca8-4e1b-b34b-0e916c6c4e8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:54.607347 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-e0a8178b-5ca8-4e1b-b34b-0e916c6c4e8d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:54.611045 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 996/1983] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:54 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:54.612155 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b1b9e9-e321-4a60-ab86-a3bb3939bc14 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:54.612515 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b1b9e9-e321-4a60-ab86-a3bb3939bc14 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:54.613056 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b1b9e9-e321-4a60-ab86-a3bb3939bc14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:54.628250 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a8178b-5ca8-4e1b-b34b-0e916c6c4e8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:54.629870 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a8178b-5ca8-4e1b-b34b-0e916c6c4e8d service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:54.637117 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-25b1b9e9-e321-4a60-ab86-a3bb3939bc14 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:54.638939 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-25b1b9e9-e321-4a60-ab86-a3bb3939bc14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:54.639613 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 988/1984] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:31:54 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:31:54.640784 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a8178b-5ca8-4e1b-b34b-0e916c6c4e8d service neutron] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:54.650279 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a8178b-5ca8-4e1b-b34b-0e916c6c4e8d service neutron] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:54.652818 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 997/1985] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:54 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:54.653961 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b1b9e9-e321-4a60-ab86-a3bb3939bc14 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:54.663265 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3ec860-c1b0-4b34-a1fe-66f98ec75766 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:54.664013 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3ec860-c1b0-4b34-a1fe-66f98ec75766 service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:54.664299 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3ec860-c1b0-4b34-a1fe-66f98ec75766 service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:54.664573 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3ec860-c1b0-4b34-a1fe-66f98ec75766 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:54.667162 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b1b9e9-e321-4a60-ab86-a3bb3939bc14 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:54.673897 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6866c6-c550-4f4d-9862-98a4a27eaf3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:54.674757 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6866c6-c550-4f4d-9862-98a4a27eaf3a admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:54.675199 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6866c6-c550-4f4d-9862-98a4a27eaf3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:54.675620 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6866c6-c550-4f4d-9862-98a4a27eaf3a admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:54.676261 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3ec860-c1b0-4b34-a1fe-66f98ec75766 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:54.676598 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6866c6-c550-4f4d-9862-98a4a27eaf3a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:31:54.677179 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6866c6-c550-4f4d-9862-98a4a27eaf3a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:54.678545 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3ec860-c1b0-4b34-a1fe-66f98ec75766 service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:54.679801 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6866c6-c550-4f4d-9862-98a4a27eaf3a admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:54.690027 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 989/1986] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:31:54 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:31:54.691450 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74d5c74-397e-4983-8820-7a6a24d0bb51 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:54.692032 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74d5c74-397e-4983-8820-7a6a24d0bb51 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:54.692358 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74d5c74-397e-4983-8820-7a6a24d0bb51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:54.695981 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 998/1987] 10.222.0.102 () {58 vars in 1158 bytes} [Thu Sep 1 17:31:54 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:31:54.700333 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6866c6-c550-4f4d-9862-98a4a27eaf3a admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:54.704330 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:INSERT=2,keystone:DELETE=8,keystone:SELECT=90 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:54.714341 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6866c6-c550-4f4d-9862-98a4a27eaf3a admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:54.721934 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-a74d5c74-397e-4983-8820-7a6a24d0bb51 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:54.723581 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-a74d5c74-397e-4983-8820-7a6a24d0bb51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:54.729603 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8fcb65-4f86-424b-8f8b-ee3f54b47999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:54.731094 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8fcb65-4f86-424b-8f8b-ee3f54b47999 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:54.732598 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8fcb65-4f86-424b-8f8b-ee3f54b47999 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:54.734321 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 990/1988] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:54 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:54.737720 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8fcb65-4f86-424b-8f8b-ee3f54b47999 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:54.739617 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8fcb65-4f86-424b-8f8b-ee3f54b47999 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:54.740580 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74d5c74-397e-4983-8820-7a6a24d0bb51 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:54.744302 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8fcb65-4f86-424b-8f8b-ee3f54b47999 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:54.765255 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74d5c74-397e-4983-8820-7a6a24d0bb51 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:54.771981 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7969a5-f5ea-4331-8342-884c67a5a28b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:54.772780 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7969a5-f5ea-4331-8342-884c67a5a28b admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:54.773155 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7969a5-f5ea-4331-8342-884c67a5a28b admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:54.773951 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7969a5-f5ea-4331-8342-884c67a5a28b admin admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:54.774837 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7969a5-f5ea-4331-8342-884c67a5a28b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:31:54.775137 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7969a5-f5ea-4331-8342-884c67a5a28b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:54.776845 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7969a5-f5ea-4331-8342-884c67a5a28b admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:54.785089 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 999/1989] 10.222.0.102 () {60 vars in 1158 bytes} [Thu Sep 1 17:31:54 2022] POST /identity/v3/projects => generated 370 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:31:54.790066 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8fcb65-4f86-424b-8f8b-ee3f54b47999 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:54.824591 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 991/1990] 10.222.0.102 () {58 vars in 1158 bytes} [Thu Sep 1 17:31:54 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:31:54.826007 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7969a5-f5ea-4331-8342-884c67a5a28b admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:54.847900 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7969a5-f5ea-4331-8342-884c67a5a28b admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:54.911525 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8fcb65-4f86-424b-8f8b-ee3f54b47999 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:54.918370 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce40736-1b94-4b19-872d-fddc9c4cf0d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:54.920248 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce40736-1b94-4b19-872d-fddc9c4cf0d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:54.920248 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce40736-1b94-4b19-872d-fddc9c4cf0d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:54.920453 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce40736-1b94-4b19-872d-fddc9c4cf0d7 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:54.920988 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce40736-1b94-4b19-872d-fddc9c4cf0d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:54.923127 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce40736-1b94-4b19-872d-fddc9c4cf0d7 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:54.945975 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb92fea3-8497-47f1-aa32-24150ac0df31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:54.947492 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb92fea3-8497-47f1-aa32-24150ac0df31 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:54.947492 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb92fea3-8497-47f1-aa32-24150ac0df31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:54.947492 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb92fea3-8497-47f1-aa32-24150ac0df31 admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:54.948053 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb92fea3-8497-47f1-aa32-24150ac0df31 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:54.950947 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb92fea3-8497-47f1-aa32-24150ac0df31 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:54.951242 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1000/1991] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:31:54 2022] POST /identity/v3/users => generated 286 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:31:54.956254 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce40736-1b94-4b19-872d-fddc9c4cf0d7 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:54.968728 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce40736-1b94-4b19-872d-fddc9c4cf0d7 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:54.974621 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf581879-efbc-4d53-bcd9-7a778c1ac76e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:54.975368 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf581879-efbc-4d53-bcd9-7a778c1ac76e admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:54.975828 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf581879-efbc-4d53-bcd9-7a778c1ac76e admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:54.976923 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf581879-efbc-4d53-bcd9-7a778c1ac76e admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:54.976923 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf581879-efbc-4d53-bcd9-7a778c1ac76e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:54.978781 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf581879-efbc-4d53-bcd9-7a778c1ac76e admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:54.985472 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 992/1992] 10.222.0.102 () {60 vars in 1158 bytes} [Thu Sep 1 17:31:54 2022] POST /identity/v3/projects => generated 360 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:31:54.987352 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb92fea3-8497-47f1-aa32-24150ac0df31 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:54.992588 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1001/1993] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:31:54 2022] GET /identity/v3/roles => generated 1427 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:31:54.996240 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf581879-efbc-4d53-bcd9-7a778c1ac76e admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:55.021837 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf581879-efbc-4d53-bcd9-7a778c1ac76e admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:55.084423 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb92fea3-8497-47f1-aa32-24150ac0df31 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:55.093172 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9be203-e651-422d-b857-4f6b6d93ace1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:55.094212 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9be203-e651-422d-b857-4f6b6d93ace1 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:55.094492 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9be203-e651-422d-b857-4f6b6d93ace1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:55.095114 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9be203-e651-422d-b857-4f6b6d93ace1 admin admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:55.096439 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9be203-e651-422d-b857-4f6b6d93ace1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:55.098954 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9be203-e651-422d-b857-4f6b6d93ace1 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:55.108955 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f3a5da-0222-4bd5-ada8-133e28eebaf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:55.109652 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f3a5da-0222-4bd5-ada8-133e28eebaf0 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:55.109886 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f3a5da-0222-4bd5-ada8-133e28eebaf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:55.110202 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f3a5da-0222-4bd5-ada8-133e28eebaf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:55.110847 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f3a5da-0222-4bd5-ada8-133e28eebaf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:55.113120 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f3a5da-0222-4bd5-ada8-133e28eebaf0 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:55.126651 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1002/1994] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:31:54 2022] GET /identity/v3/roles => generated 1427 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 17:31:55.130426 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f3a5da-0222-4bd5-ada8-133e28eebaf0 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:55.145039 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f3a5da-0222-4bd5-ada8-133e28eebaf0 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:55.151086 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 993/1995] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:31:54 2022] POST /identity/v3/users => generated 281 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:31:55.155443 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9be203-e651-422d-b857-4f6b6d93ace1 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:55.157669 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe3c612-40cd-462c-9e2c-1fa6e3edf6d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:55.158465 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe3c612-40cd-462c-9e2c-1fa6e3edf6d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:55.158879 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe3c612-40cd-462c-9e2c-1fa6e3edf6d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:55.159768 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe3c612-40cd-462c-9e2c-1fa6e3edf6d9 admin admin] PATH_INFO: `/v3/projects/98d5dfc87a214d0fbc086b9eff7a5bf2/users/1c2839729b5c4814b80113e9df32aade/roles/b75cb360925e4ba5a65f14965c103ffd` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:55.173121 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9be203-e651-422d-b857-4f6b6d93ace1 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:55.178217 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe3c612-40cd-462c-9e2c-1fa6e3edf6d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98d5dfc87a214d0fbc086b9eff7a5bf2, user_id=1c2839729b5c4814b80113e9df32aade, role_id=b75cb360925e4ba5a65f14965c103ffd)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:55.179997 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0daf5d-c790-45bb-810d-3aa62a3c10dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:55.180998 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0daf5d-c790-45bb-810d-3aa62a3c10dc admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:55.181413 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe3c612-40cd-462c-9e2c-1fa6e3edf6d9 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:55.181625 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0daf5d-c790-45bb-810d-3aa62a3c10dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:55.182397 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0daf5d-c790-45bb-810d-3aa62a3c10dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:55.183537 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0daf5d-c790-45bb-810d-3aa62a3c10dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:55.186849 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0daf5d-c790-45bb-810d-3aa62a3c10dc admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:55.209013 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 994/1996] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:31:55 2022] GET /identity/v3/roles => generated 1427 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 17:31:55.209751 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0daf5d-c790-45bb-810d-3aa62a3c10dc admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:55.214537 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:55.229782 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1003/1997] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:31:55 2022] PUT /identity/v3/projects/98d5dfc87a214d0fbc086b9eff7a5bf2/users/1c2839729b5c4814b80113e9df32aade/roles/b75cb360925e4ba5a65f14965c103ffd => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:31:55.239185 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698c1cd3-4063-44b0-811c-da005a56c3fa None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:55.239457 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0daf5d-c790-45bb-810d-3aa62a3c10dc admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:55.240175 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698c1cd3-4063-44b0-811c-da005a56c3fa None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:55.243072 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698c1cd3-4063-44b0-811c-da005a56c3fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:55.311330 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-698c1cd3-4063-44b0-811c-da005a56c3fa None None] MFA Rules not processed for user `1c2839729b5c4814b80113e9df32aade`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:55.311330 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-698c1cd3-4063-44b0-811c-da005a56c3fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:55.344524 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:55.371396 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d035df81-667b-4573-9dcc-2326cde6fe39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:55.372749 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d035df81-667b-4573-9dcc-2326cde6fe39 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:55.373963 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d035df81-667b-4573-9dcc-2326cde6fe39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:55.374428 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d035df81-667b-4573-9dcc-2326cde6fe39 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:55.375358 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d035df81-667b-4573-9dcc-2326cde6fe39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:55.378081 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d035df81-667b-4573-9dcc-2326cde6fe39 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:55.393760 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 995/1998] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:31:55 2022] GET /identity/v3/roles => generated 1427 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 17:31:55.394962 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d035df81-667b-4573-9dcc-2326cde6fe39 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:55.396706 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-d035df81-667b-4573-9dcc-2326cde6fe39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:55.407326 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1004/1999] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:55 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:55.408386 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698c1cd3-4063-44b0-811c-da005a56c3fa None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:55.409887 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-698c1cd3-4063-44b0-811c-da005a56c3fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:55.411640 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d035df81-667b-4573-9dcc-2326cde6fe39 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:55.417256 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56a8891-83de-43bb-8fd8-537c150727b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:55.418752 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56a8891-83de-43bb-8fd8-537c150727b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:55.419067 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56a8891-83de-43bb-8fd8-537c150727b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:55.419342 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56a8891-83de-43bb-8fd8-537c150727b5 admin admin] PATH_INFO: `/v3/projects/ebcc9a93cec94f15a80d34cae26f661e` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:55.421562 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56a8891-83de-43bb-8fd8-537c150727b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ebcc9a93cec94f15a80d34cae26f661e)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:55.423432 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56a8891-83de-43bb-8fd8-537c150727b5 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:55.424209 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698c1cd3-4063-44b0-811c-da005a56c3fa None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:55.429642 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8e1587-45d1-4f59-85a6-8f6b9e7b890f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:55.430293 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8e1587-45d1-4f59-85a6-8f6b9e7b890f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:55.430567 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8e1587-45d1-4f59-85a6-8f6b9e7b890f admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:55.430847 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8e1587-45d1-4f59-85a6-8f6b9e7b890f admin admin] PATH_INFO: `/v3/projects/9f61bc7db6a24f5c990e875a5fed61e3` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:55.432000 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8e1587-45d1-4f59-85a6-8f6b9e7b890f admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f61bc7db6a24f5c990e875a5fed61e3)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:55.434143 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8e1587-45d1-4f59-85a6-8f6b9e7b890f admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:55.487300 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-c56a8891-83de-43bb-8fd8-537c150727b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ebcc9a93cec94f15a80d34cae26f661e', 'request_id': 'req-c56a8891-83de-43bb-8fd8-537c150727b5'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:55.496360 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-c56a8891-83de-43bb-8fd8-537c150727b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ebcc9a93cec94f15a80d34cae26f661e', 'request_id': 'req-c56a8891-83de-43bb-8fd8-537c150727b5'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:55.498644 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-2b8e1587-45d1-4f59-85a6-8f6b9e7b890f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f61bc7db6a24f5c990e875a5fed61e3', 'request_id': 'req-2b8e1587-45d1-4f59-85a6-8f6b9e7b890f'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:55.519184 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-2b8e1587-45d1-4f59-85a6-8f6b9e7b890f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f61bc7db6a24f5c990e875a5fed61e3', 'request_id': 'req-2b8e1587-45d1-4f59-85a6-8f6b9e7b890f'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:55.520168 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-c56a8891-83de-43bb-8fd8-537c150727b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ebcc9a93cec94f15a80d34cae26f661e', 'request_id': 'req-c56a8891-83de-43bb-8fd8-537c150727b5'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:55.530785 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-2b8e1587-45d1-4f59-85a6-8f6b9e7b890f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f61bc7db6a24f5c990e875a5fed61e3', 'request_id': 'req-2b8e1587-45d1-4f59-85a6-8f6b9e7b890f'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:55.546790 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-c56a8891-83de-43bb-8fd8-537c150727b5 admin admin] The token cache is being invalidate because project ebcc9a93cec94f15a80d34cae26f661e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:31:55.547044 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-c56a8891-83de-43bb-8fd8-537c150727b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:55.547821 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-c56a8891-83de-43bb-8fd8-537c150727b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:55.549609 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 996/2000] 10.222.0.102 () {58 vars in 1238 bytes} [Thu Sep 1 17:31:55 2022] DELETE /identity/v3/projects/ebcc9a93cec94f15a80d34cae26f661e => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:55.551118 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-2b8e1587-45d1-4f59-85a6-8f6b9e7b890f admin admin] The token cache is being invalidate because project 9f61bc7db6a24f5c990e875a5fed61e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:31:55.551550 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-2b8e1587-45d1-4f59-85a6-8f6b9e7b890f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:55.554534 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56a8891-83de-43bb-8fd8-537c150727b5 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:55.554865 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-2b8e1587-45d1-4f59-85a6-8f6b9e7b890f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:55.555196 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-c56a8891-83de-43bb-8fd8-537c150727b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:55.557418 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1005/2001] 10.222.0.102 () {58 vars in 1238 bytes} [Thu Sep 1 17:31:55 2022] DELETE /identity/v3/projects/9f61bc7db6a24f5c990e875a5fed61e3 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:55.557418 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8e1587-45d1-4f59-85a6-8f6b9e7b890f admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:55.558583 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-2b8e1587-45d1-4f59-85a6-8f6b9e7b890f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:55.628223 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56a8891-83de-43bb-8fd8-537c150727b5 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:55.630082 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8e1587-45d1-4f59-85a6-8f6b9e7b890f admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:55.633497 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111fb4f1-ffc3-4d7d-a0a7-165b0e3192b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:55.634163 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111fb4f1-ffc3-4d7d-a0a7-165b0e3192b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:55.634443 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111fb4f1-ffc3-4d7d-a0a7-165b0e3192b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:55.634719 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111fb4f1-ffc3-4d7d-a0a7-165b0e3192b7 admin admin] PATH_INFO: `/v3/projects/349c2a93a2b6418da6081ebd3042cb96/users/5d3e7bfe19944b449d2dcae630639e52/roles/b75cb360925e4ba5a65f14965c103ffd` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:55.636036 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a33136e-fb73-42bf-b377-dfc019e97e59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:55.636952 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a33136e-fb73-42bf-b377-dfc019e97e59 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:55.637472 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a33136e-fb73-42bf-b377-dfc019e97e59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:55.639173 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a33136e-fb73-42bf-b377-dfc019e97e59 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:55.639965 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a33136e-fb73-42bf-b377-dfc019e97e59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:55.642609 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a33136e-fb73-42bf-b377-dfc019e97e59 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:55.652617 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111fb4f1-ffc3-4d7d-a0a7-165b0e3192b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=349c2a93a2b6418da6081ebd3042cb96, user_id=5d3e7bfe19944b449d2dcae630639e52, role_id=b75cb360925e4ba5a65f14965c103ffd)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:55.655450 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111fb4f1-ffc3-4d7d-a0a7-165b0e3192b7 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:55.673113 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:55.681993 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 997/2002] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:31:55 2022] PUT /identity/v3/projects/349c2a93a2b6418da6081ebd3042cb96/users/5d3e7bfe19944b449d2dcae630639e52/roles/b75cb360925e4ba5a65f14965c103ffd => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:55.683356 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b66555-7c5a-4817-ac73-79ae0af35e98 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:55.683647 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b66555-7c5a-4817-ac73-79ae0af35e98 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:55.685549 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1006/2003] 10.222.0.102 () {60 vars in 1158 bytes} [Thu Sep 1 17:31:55 2022] POST /identity/v3/projects => generated 370 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:31:55.685661 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b66555-7c5a-4817-ac73-79ae0af35e98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:55.686471 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a33136e-fb73-42bf-b377-dfc019e97e59 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:55.688113 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-9a33136e-fb73-42bf-b377-dfc019e97e59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:55.705369 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-25b66555-7c5a-4817-ac73-79ae0af35e98 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:55.706629 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-25b66555-7c5a-4817-ac73-79ae0af35e98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:55.763741 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:55.778316 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a33136e-fb73-42bf-b377-dfc019e97e59 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:55.783755 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 998/2004] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:55 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:55.784655 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b66555-7c5a-4817-ac73-79ae0af35e98 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:55.784852 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1352f281-ed54-4ef8-a334-79689740459b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:55.785818 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1352f281-ed54-4ef8-a334-79689740459b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:55.786251 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1352f281-ed54-4ef8-a334-79689740459b admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:55.787283 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1352f281-ed54-4ef8-a334-79689740459b admin admin] PATH_INFO: `/v3/users/ace575ade14541cc8ae4bd03d48acebc` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:55.788871 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1352f281-ed54-4ef8-a334-79689740459b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ace575ade14541cc8ae4bd03d48acebc)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:55.791304 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1352f281-ed54-4ef8-a334-79689740459b admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:55.795518 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b66555-7c5a-4817-ac73-79ae0af35e98 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:55.802215 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b2142c-9dfc-4b47-b1d2-3248c0232187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:55.802835 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b2142c-9dfc-4b47-b1d2-3248c0232187 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:55.803124 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b2142c-9dfc-4b47-b1d2-3248c0232187 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:55.803401 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b2142c-9dfc-4b47-b1d2-3248c0232187 admin admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:55.804041 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b2142c-9dfc-4b47-b1d2-3248c0232187 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:55.806207 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b2142c-9dfc-4b47-b1d2-3248c0232187 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:55.833516 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 999/2005] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:31:55 2022] POST /identity/v3/users => generated 292 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:31:55.835242 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13e9c7c-03e6-4cf4-81af-98b748ca4cc9 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:55.835583 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13e9c7c-03e6-4cf4-81af-98b748ca4cc9 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:55.835913 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13e9c7c-03e6-4cf4-81af-98b748ca4cc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:55.851628 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-1352f281-ed54-4ef8-a334-79689740459b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ace575ade14541cc8ae4bd03d48acebc'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:55.864290 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-1352f281-ed54-4ef8-a334-79689740459b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ace575ade14541cc8ae4bd03d48acebc'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:55.866972 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-a13e9c7c-03e6-4cf4-81af-98b748ca4cc9 None None] MFA Rules not processed for user `5d3e7bfe19944b449d2dcae630639e52`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:55.868360 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-a13e9c7c-03e6-4cf4-81af-98b748ca4cc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:55.871346 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-1352f281-ed54-4ef8-a334-79689740459b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ace575ade14541cc8ae4bd03d48acebc'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:55.872707 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-1352f281-ed54-4ef8-a334-79689740459b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ace575ade14541cc8ae4bd03d48acebc'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:55.873426 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-1352f281-ed54-4ef8-a334-79689740459b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ace575ade14541cc8ae4bd03d48acebc'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:55.886017 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1007/2006] 10.222.0.102 () {58 vars in 1229 bytes} [Thu Sep 1 17:31:55 2022] DELETE /identity/v3/users/ace575ade14541cc8ae4bd03d48acebc => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:55.889985 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf66d7e-d166-491e-a513-9a9612ca5b5b None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:55.890276 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf66d7e-d166-491e-a513-9a9612ca5b5b None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:55.890557 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf66d7e-d166-491e-a513-9a9612ca5b5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:55.913144 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-baf66d7e-d166-491e-a513-9a9612ca5b5b None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:55.914419 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-baf66d7e-d166-491e-a513-9a9612ca5b5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:55.972075 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1000/2007] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:55 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:55.974578 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13e9c7c-03e6-4cf4-81af-98b748ca4cc9 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:55.975835 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-a13e9c7c-03e6-4cf4-81af-98b748ca4cc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:56.002780 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1008/2008] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:55 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:56.003954 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13e9c7c-03e6-4cf4-81af-98b748ca4cc9 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:56.007035 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf66d7e-d166-491e-a513-9a9612ca5b5b None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:56.017233 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf66d7e-d166-491e-a513-9a9612ca5b5b None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:56.026379 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e22801d-fa0f-4125-bd18-43964fe0515b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:56.027097 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e22801d-fa0f-4125-bd18-43964fe0515b admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:56.027483 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e22801d-fa0f-4125-bd18-43964fe0515b admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:56.027760 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e22801d-fa0f-4125-bd18-43964fe0515b admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:56.028485 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e22801d-fa0f-4125-bd18-43964fe0515b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:56.030207 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e22801d-fa0f-4125-bd18-43964fe0515b admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:56.030326 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1559593b-96e2-4cae-82ce-531fe9d0af6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:56.031188 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1559593b-96e2-4cae-82ce-531fe9d0af6f admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:56.032403 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1559593b-96e2-4cae-82ce-531fe9d0af6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:56.032784 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1559593b-96e2-4cae-82ce-531fe9d0af6f admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:56.034108 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1559593b-96e2-4cae-82ce-531fe9d0af6f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:31:56.035118 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1559593b-96e2-4cae-82ce-531fe9d0af6f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:56.037587 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1559593b-96e2-4cae-82ce-531fe9d0af6f admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:56.050484 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1001/2009] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:31:55 2022] GET /identity/v3/roles => generated 1427 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:31:56.051726 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e22801d-fa0f-4125-bd18-43964fe0515b admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:56.053991 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-9e22801d-fa0f-4125-bd18-43964fe0515b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:56.067222 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1009/2010] 10.222.0.102 () {58 vars in 1158 bytes} [Thu Sep 1 17:31:56 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:31:56.070355 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1559593b-96e2-4cae-82ce-531fe9d0af6f admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:56.078984 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:56.086489 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1559593b-96e2-4cae-82ce-531fe9d0af6f admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:56.096898 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec18176-99b8-4195-a8f1-e0977cfd192c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:56.098324 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec18176-99b8-4195-a8f1-e0977cfd192c admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:56.099373 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec18176-99b8-4195-a8f1-e0977cfd192c admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:56.102563 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec18176-99b8-4195-a8f1-e0977cfd192c admin admin] PATH_INFO: `/v3/projects/66970187da3d4d56938d16397d7708c5/users/cfceddaf9335487f990f1a32bf18e97e/roles/36577a6f46d642d4b81805ede9872490` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:56.125511 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec18176-99b8-4195-a8f1-e0977cfd192c admin admin] RBAC: Authorizing `identity:create_grant(project_id=66970187da3d4d56938d16397d7708c5, user_id=cfceddaf9335487f990f1a32bf18e97e, role_id=36577a6f46d642d4b81805ede9872490)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:56.129253 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec18176-99b8-4195-a8f1-e0977cfd192c admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:56.152542 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1010/2011] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:31:56 2022] PUT /identity/v3/projects/66970187da3d4d56938d16397d7708c5/users/cfceddaf9335487f990f1a32bf18e97e/roles/36577a6f46d642d4b81805ede9872490 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:56.154384 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec18176-99b8-4195-a8f1-e0977cfd192c admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:56.161061 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e22801d-fa0f-4125-bd18-43964fe0515b admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:56.226985 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec18176-99b8-4195-a8f1-e0977cfd192c admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:56.234186 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b734c105-63c5-472d-b6e4-79957158c383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:56.234810 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b734c105-63c5-472d-b6e4-79957158c383 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:56.235079 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b734c105-63c5-472d-b6e4-79957158c383 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:56.235350 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b734c105-63c5-472d-b6e4-79957158c383 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:56.235986 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b734c105-63c5-472d-b6e4-79957158c383 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:56.237418 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c43e0e-5bbc-4bf9-8cf9-49279d7ee75e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '523ed589872a45a09b6a7675a7fb0b75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:56.238154 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c43e0e-5bbc-4bf9-8cf9-49279d7ee75e service cinder] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:56.238580 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c43e0e-5bbc-4bf9-8cf9-49279d7ee75e service cinder] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:56.238784 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c43e0e-5bbc-4bf9-8cf9-49279d7ee75e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:56.241373 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b734c105-63c5-472d-b6e4-79957158c383 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:56.304535 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:56.307545 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1011/2012] 10.222.0.102 () {60 vars in 1158 bytes} [Thu Sep 1 17:31:56 2022] POST /identity/v3/projects => generated 376 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:31:56.308908 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b734c105-63c5-472d-b6e4-79957158c383 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:56.405666 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c43e0e-5bbc-4bf9-8cf9-49279d7ee75e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:56.406455 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b734c105-63c5-472d-b6e4-79957158c383 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:56.407211 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c43e0e-5bbc-4bf9-8cf9-49279d7ee75e service cinder] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:56.416169 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1a6051-4f3d-46c8-b2cd-435c940d6964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:56.416798 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1a6051-4f3d-46c8-b2cd-435c940d6964 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:56.417105 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1a6051-4f3d-46c8-b2cd-435c940d6964 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:56.417399 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1a6051-4f3d-46c8-b2cd-435c940d6964 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:56.418412 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1a6051-4f3d-46c8-b2cd-435c940d6964 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:56.420879 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1a6051-4f3d-46c8-b2cd-435c940d6964 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:56.431792 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1002/2013] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:31:56 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:31:56.436342 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1012/2014] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:31:56 2022] GET /identity/v3/roles => generated 1427 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:31:56.436722 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1a6051-4f3d-46c8-b2cd-435c940d6964 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:56.439633 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-3f1a6051-4f3d-46c8-b2cd-435c940d6964 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:56.443993 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c43e0e-5bbc-4bf9-8cf9-49279d7ee75e service cinder] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:56.463087 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1a6051-4f3d-46c8-b2cd-435c940d6964 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:56.467438 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c43e0e-5bbc-4bf9-8cf9-49279d7ee75e service cinder] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:56.474143 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fb8448-6a65-467d-890e-c476bcbe617e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:56.475102 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fb8448-6a65-467d-890e-c476bcbe617e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:56.475529 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fb8448-6a65-467d-890e-c476bcbe617e admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:56.475924 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fb8448-6a65-467d-890e-c476bcbe617e admin admin] PATH_INFO: `/v3/projects/c37f6b8752bb445cbef2eb2a510aba66` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:56.476035 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2989cf-6813-43aa-918e-8671f82cc0e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:56.476738 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2989cf-6813-43aa-918e-8671f82cc0e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:56.477026 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2989cf-6813-43aa-918e-8671f82cc0e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:56.477334 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2989cf-6813-43aa-918e-8671f82cc0e6 admin admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:56.477524 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fb8448-6a65-467d-890e-c476bcbe617e admin admin] RBAC: Authorizing `identity:delete_project(project_id=c37f6b8752bb445cbef2eb2a510aba66)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:56.478034 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2989cf-6813-43aa-918e-8671f82cc0e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:56.479660 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fb8448-6a65-467d-890e-c476bcbe617e admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:56.479965 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2989cf-6813-43aa-918e-8671f82cc0e6 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:56.515799 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1003/2015] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:31:56 2022] POST /identity/v3/users => generated 289 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:31:56.515799 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2989cf-6813-43aa-918e-8671f82cc0e6 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:56.531381 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2989cf-6813-43aa-918e-8671f82cc0e6 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:56.539728 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e716f608-19f1-4210-8199-253714503337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:56.540409 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e716f608-19f1-4210-8199-253714503337 admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:56.540930 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e716f608-19f1-4210-8199-253714503337 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:56.541211 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e716f608-19f1-4210-8199-253714503337 admin admin] PATH_INFO: `/v3/projects/66970187da3d4d56938d16397d7708c5/users/cfceddaf9335487f990f1a32bf18e97e/roles/36577a6f46d642d4b81805ede9872490` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:56.545209 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e716f608-19f1-4210-8199-253714503337 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66970187da3d4d56938d16397d7708c5, user_id=cfceddaf9335487f990f1a32bf18e97e, role_id=36577a6f46d642d4b81805ede9872490)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:56.547897 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e716f608-19f1-4210-8199-253714503337 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:56.558877 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1004/2016] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:31:56 2022] PUT /identity/v3/projects/66970187da3d4d56938d16397d7708c5/users/cfceddaf9335487f990f1a32bf18e97e/roles/36577a6f46d642d4b81805ede9872490 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:56.569754 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e716f608-19f1-4210-8199-253714503337 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:56.570158 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c1fb8448-6a65-467d-890e-c476bcbe617e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c37f6b8752bb445cbef2eb2a510aba66', 'request_id': 'req-c1fb8448-6a65-467d-890e-c476bcbe617e'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:56.571613 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-e716f608-19f1-4210-8199-253714503337 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:56.579044 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c1fb8448-6a65-467d-890e-c476bcbe617e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c37f6b8752bb445cbef2eb2a510aba66', 'request_id': 'req-c1fb8448-6a65-467d-890e-c476bcbe617e'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:56.581397 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:56.594077 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c1fb8448-6a65-467d-890e-c476bcbe617e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c37f6b8752bb445cbef2eb2a510aba66', 'request_id': 'req-c1fb8448-6a65-467d-890e-c476bcbe617e'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:56.613026 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c1fb8448-6a65-467d-890e-c476bcbe617e admin admin] The token cache is being invalidate because project c37f6b8752bb445cbef2eb2a510aba66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:31:56.613380 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c1fb8448-6a65-467d-890e-c476bcbe617e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:56.614044 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c1fb8448-6a65-467d-890e-c476bcbe617e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:56.615660 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1013/2017] 10.222.0.102 () {58 vars in 1238 bytes} [Thu Sep 1 17:31:56 2022] DELETE /identity/v3/projects/c37f6b8752bb445cbef2eb2a510aba66 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:56.616497 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fb8448-6a65-467d-890e-c476bcbe617e admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:56.617960 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-c1fb8448-6a65-467d-890e-c476bcbe617e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:56.662282 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e716f608-19f1-4210-8199-253714503337 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:56.666982 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712aefcd-af07-4e81-84b4-eaea06154a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:56.667611 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712aefcd-af07-4e81-84b4-eaea06154a6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:56.667890 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712aefcd-af07-4e81-84b4-eaea06154a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:56.668162 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712aefcd-af07-4e81-84b4-eaea06154a6b admin admin] PATH_INFO: `/v3/users/5d3e7bfe19944b449d2dcae630639e52` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:56.669387 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712aefcd-af07-4e81-84b4-eaea06154a6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d3e7bfe19944b449d2dcae630639e52)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:56.671300 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712aefcd-af07-4e81-84b4-eaea06154a6b admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:56.695514 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fb8448-6a65-467d-890e-c476bcbe617e admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:56.705160 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b58058-e7d6-4bc7-b3d5-6a555643929f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:56.705753 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b58058-e7d6-4bc7-b3d5-6a555643929f admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:56.706247 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b58058-e7d6-4bc7-b3d5-6a555643929f admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:56.706565 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b58058-e7d6-4bc7-b3d5-6a555643929f admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:56.707378 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b58058-e7d6-4bc7-b3d5-6a555643929f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:56.709010 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b58058-e7d6-4bc7-b3d5-6a555643929f admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:56.728637 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1014/2018] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:31:56 2022] GET /identity/v3/roles => generated 1427 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:31:56.730976 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75228d37-421b-4e09-8f29-e23f4e39e294 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:56.731318 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75228d37-421b-4e09-8f29-e23f4e39e294 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:56.731668 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75228d37-421b-4e09-8f29-e23f4e39e294 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:56.741596 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-712aefcd-af07-4e81-84b4-eaea06154a6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d3e7bfe19944b449d2dcae630639e52'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:56.744138 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-712aefcd-af07-4e81-84b4-eaea06154a6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d3e7bfe19944b449d2dcae630639e52'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:56.761998 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-712aefcd-af07-4e81-84b4-eaea06154a6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d3e7bfe19944b449d2dcae630639e52'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:56.772583 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-75228d37-421b-4e09-8f29-e23f4e39e294 None None] MFA Rules not processed for user `cfceddaf9335487f990f1a32bf18e97e`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:56.774188 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-75228d37-421b-4e09-8f29-e23f4e39e294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:56.782245 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-712aefcd-af07-4e81-84b4-eaea06154a6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d3e7bfe19944b449d2dcae630639e52'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:56.783540 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-712aefcd-af07-4e81-84b4-eaea06154a6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d3e7bfe19944b449d2dcae630639e52'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:56.800675 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1005/2019] 10.222.0.102 () {58 vars in 1229 bytes} [Thu Sep 1 17:31:56 2022] DELETE /identity/v3/users/5d3e7bfe19944b449d2dcae630639e52 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:56.801713 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712aefcd-af07-4e81-84b4-eaea06154a6b admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:56.804541 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-712aefcd-af07-4e81-84b4-eaea06154a6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:56.818051 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:56.905021 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1015/2020] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:56 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:56.906125 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75228d37-421b-4e09-8f29-e23f4e39e294 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:56.910134 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-75228d37-421b-4e09-8f29-e23f4e39e294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:56.920680 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712aefcd-af07-4e81-84b4-eaea06154a6b admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:56.925080 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75228d37-421b-4e09-8f29-e23f4e39e294 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:56.925670 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cdc8db-0970-408b-97d9-cae089985c37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:56.926418 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cdc8db-0970-408b-97d9-cae089985c37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:56.926695 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cdc8db-0970-408b-97d9-cae089985c37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:56.926973 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cdc8db-0970-408b-97d9-cae089985c37 admin admin] PATH_INFO: `/v3/users/278aa08de8ea41aeb477218d757dcd53` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:56.928138 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cdc8db-0970-408b-97d9-cae089985c37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=278aa08de8ea41aeb477218d757dcd53)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:56.930312 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cdc8db-0970-408b-97d9-cae089985c37 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:56.930676 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cbb6a8-dcdf-460e-8342-060c81a963fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:56.931392 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cbb6a8-dcdf-460e-8342-060c81a963fb admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:56.931879 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cbb6a8-dcdf-460e-8342-060c81a963fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:56.932151 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cbb6a8-dcdf-460e-8342-060c81a963fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:56.932912 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cbb6a8-dcdf-460e-8342-060c81a963fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:56.936010 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cbb6a8-dcdf-460e-8342-060c81a963fb admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:56.947879 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1016/2021] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:31:56 2022] GET /identity/v3/roles => generated 1427 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:31:56.949157 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7986da8-c014-4595-813c-5be9e840d80c None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:56.949564 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7986da8-c014-4595-813c-5be9e840d80c None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:56.950523 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7986da8-c014-4595-813c-5be9e840d80c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:56.977779 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-b7986da8-c014-4595-813c-5be9e840d80c None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:56.979015 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-b7986da8-c014-4595-813c-5be9e840d80c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:56.989697 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-c7cdc8db-0970-408b-97d9-cae089985c37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '278aa08de8ea41aeb477218d757dcd53'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:56.990383 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-c7cdc8db-0970-408b-97d9-cae089985c37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '278aa08de8ea41aeb477218d757dcd53'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:56.992535 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1017/2022] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:56 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:56.993703 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7986da8-c014-4595-813c-5be9e840d80c None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:56.995165 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-b7986da8-c014-4595-813c-5be9e840d80c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:57.012292 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-c7cdc8db-0970-408b-97d9-cae089985c37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '278aa08de8ea41aeb477218d757dcd53'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:57.025121 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-c7cdc8db-0970-408b-97d9-cae089985c37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '278aa08de8ea41aeb477218d757dcd53'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:57.025874 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-c7cdc8db-0970-408b-97d9-cae089985c37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '278aa08de8ea41aeb477218d757dcd53'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:57.049640 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1006/2023] 10.222.0.102 () {58 vars in 1229 bytes} [Thu Sep 1 17:31:56 2022] DELETE /identity/v3/users/278aa08de8ea41aeb477218d757dcd53 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:57.052973 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cdc8db-0970-408b-97d9-cae089985c37 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:57.054762 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:57.055858 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-c7cdc8db-0970-408b-97d9-cae089985c37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:57.058936 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7986da8-c014-4595-813c-5be9e840d80c None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:57.144987 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b0c8d0-8817-45ec-9d99-7febff3bc795 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '523ed589872a45a09b6a7675a7fb0b75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:57.146834 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b0c8d0-8817-45ec-9d99-7febff3bc795 service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:57.147884 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b0c8d0-8817-45ec-9d99-7febff3bc795 service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:57.149368 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b0c8d0-8817-45ec-9d99-7febff3bc795 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:57.154118 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-31b0c8d0-8817-45ec-9d99-7febff3bc795 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:57.155938 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cdc8db-0970-408b-97d9-cae089985c37 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:57.161143 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700761d8-736f-4273-bd19-18e8d814a263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:57.162060 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700761d8-736f-4273-bd19-18e8d814a263 admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:57.162338 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700761d8-736f-4273-bd19-18e8d814a263 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:57.162610 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700761d8-736f-4273-bd19-18e8d814a263 admin admin] PATH_INFO: `/v3/projects/46f445ab9e364c44a1e45c25f6ec679a/users/5e33ed91311241a3aec54a46880abab3/roles/b75cb360925e4ba5a65f14965c103ffd` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:57.182801 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700761d8-736f-4273-bd19-18e8d814a263 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46f445ab9e364c44a1e45c25f6ec679a, user_id=5e33ed91311241a3aec54a46880abab3, role_id=b75cb360925e4ba5a65f14965c103ffd)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:57.186913 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700761d8-736f-4273-bd19-18e8d814a263 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:57.207160 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1007/2024] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:31:57 2022] PUT /identity/v3/projects/46f445ab9e364c44a1e45c25f6ec679a/users/5e33ed91311241a3aec54a46880abab3/roles/b75cb360925e4ba5a65f14965c103ffd => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:57.218531 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700761d8-736f-4273-bd19-18e8d814a263 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:57.222228 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-700761d8-736f-4273-bd19-18e8d814a263 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:57.250259 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:57.303727 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b0c8d0-8817-45ec-9d99-7febff3bc795 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:57.306028 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b0c8d0-8817-45ec-9d99-7febff3bc795 service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:57.326594 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700761d8-736f-4273-bd19-18e8d814a263 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:57.328926 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1018/2025] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:31:56 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:31:57.330742 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acfb2aa-f0af-4f09-afa5-c6a02a2f2ce5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:57.330940 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cec7e13-979d-44b6-88cb-c412db138396 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:57.331240 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cec7e13-979d-44b6-88cb-c412db138396 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:57.331425 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acfb2aa-f0af-4f09-afa5-c6a02a2f2ce5 service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:57.331517 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cec7e13-979d-44b6-88cb-c412db138396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:57.331600 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acfb2aa-f0af-4f09-afa5-c6a02a2f2ce5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:57.331867 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acfb2aa-f0af-4f09-afa5-c6a02a2f2ce5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:57.333886 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-4acfb2aa-f0af-4f09-afa5-c6a02a2f2ce5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:57.363649 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-2cec7e13-979d-44b6-88cb-c412db138396 None None] MFA Rules not processed for user `5e33ed91311241a3aec54a46880abab3`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:57.364969 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-2cec7e13-979d-44b6-88cb-c412db138396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:57.417913 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acfb2aa-f0af-4f09-afa5-c6a02a2f2ce5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:57.420097 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acfb2aa-f0af-4f09-afa5-c6a02a2f2ce5 service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:57.445512 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1008/2026] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:31:57 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:31:57.449364 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acfb2aa-f0af-4f09-afa5-c6a02a2f2ce5 service neutron] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:57.452083 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-4acfb2aa-f0af-4f09-afa5-c6a02a2f2ce5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:57.459008 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1019/2027] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:57 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:57.463609 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cec7e13-979d-44b6-88cb-c412db138396 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:57.468855 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acfb2aa-f0af-4f09-afa5-c6a02a2f2ce5 service neutron] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:57.483027 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62dbfd3-d849-4037-b717-54b9cb94ab10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:57.483684 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62dbfd3-d849-4037-b717-54b9cb94ab10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:57.484765 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62dbfd3-d849-4037-b717-54b9cb94ab10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:57.485065 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62dbfd3-d849-4037-b717-54b9cb94ab10 admin admin] PATH_INFO: `/v3/projects/d7eeea5e32ce483089aef76cf8097d67` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:57.485333 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cec7e13-979d-44b6-88cb-c412db138396 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:57.487056 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62dbfd3-d849-4037-b717-54b9cb94ab10 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7eeea5e32ce483089aef76cf8097d67)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:57.490079 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62dbfd3-d849-4037-b717-54b9cb94ab10 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:57.491546 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b35ad1d-7125-4bd0-82cc-1521d190a461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:57.492886 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b35ad1d-7125-4bd0-82cc-1521d190a461 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:57.493683 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b35ad1d-7125-4bd0-82cc-1521d190a461 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:57.494060 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b35ad1d-7125-4bd0-82cc-1521d190a461 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:57.494813 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b35ad1d-7125-4bd0-82cc-1521d190a461 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:57.498414 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b35ad1d-7125-4bd0-82cc-1521d190a461 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:57.548789 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1020/2028] 10.222.0.102 () {60 vars in 1158 bytes} [Thu Sep 1 17:31:57 2022] POST /identity/v3/projects => generated 374 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:31:57.556018 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b35ad1d-7125-4bd0-82cc-1521d190a461 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:57.579130 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-e62dbfd3-d849-4037-b717-54b9cb94ab10 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7eeea5e32ce483089aef76cf8097d67', 'request_id': 'req-e62dbfd3-d849-4037-b717-54b9cb94ab10'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:57.589551 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-e62dbfd3-d849-4037-b717-54b9cb94ab10 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7eeea5e32ce483089aef76cf8097d67', 'request_id': 'req-e62dbfd3-d849-4037-b717-54b9cb94ab10'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:57.599605 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:57.620597 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-e62dbfd3-d849-4037-b717-54b9cb94ab10 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7eeea5e32ce483089aef76cf8097d67', 'request_id': 'req-e62dbfd3-d849-4037-b717-54b9cb94ab10'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:57.648758 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b35ad1d-7125-4bd0-82cc-1521d190a461 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:57.663331 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-e62dbfd3-d849-4037-b717-54b9cb94ab10 admin admin] The token cache is being invalidate because project d7eeea5e32ce483089aef76cf8097d67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:31:57.663893 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-e62dbfd3-d849-4037-b717-54b9cb94ab10 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:57.664690 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-e62dbfd3-d849-4037-b717-54b9cb94ab10 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:57.666844 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1009/2029] 10.222.0.102 () {58 vars in 1238 bytes} [Thu Sep 1 17:31:57 2022] DELETE /identity/v3/projects/d7eeea5e32ce483089aef76cf8097d67 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:57.673937 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a240f9-bc32-4ffb-8d2d-6eae0bc5bd03 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:57.674608 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a240f9-bc32-4ffb-8d2d-6eae0bc5bd03 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:57.675561 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a240f9-bc32-4ffb-8d2d-6eae0bc5bd03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:57.706563 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-86a240f9-bc32-4ffb-8d2d-6eae0bc5bd03 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:57.709635 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-86a240f9-bc32-4ffb-8d2d-6eae0bc5bd03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:57.776454 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f249c8f3-d942-4293-9077-9e66da55d81f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:57.777705 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f249c8f3-d942-4293-9077-9e66da55d81f admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:57.778760 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f249c8f3-d942-4293-9077-9e66da55d81f admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:57.779067 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f249c8f3-d942-4293-9077-9e66da55d81f admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:57.779863 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f249c8f3-d942-4293-9077-9e66da55d81f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:57.782343 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f249c8f3-d942-4293-9077-9e66da55d81f admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:57.789456 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:57.869399 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1021/2030] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:31:57 2022] POST /identity/v3/users => generated 294 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:31:57.870408 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f249c8f3-d942-4293-9077-9e66da55d81f admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:57.872101 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-f249c8f3-d942-4293-9077-9e66da55d81f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:57.878932 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1010/2031] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:57 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:31:57.880365 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61093fa6-43bf-439f-aba3-92239ccc5606 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:57.880658 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61093fa6-43bf-439f-aba3-92239ccc5606 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:57.880924 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61093fa6-43bf-439f-aba3-92239ccc5606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:57.892181 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f249c8f3-d942-4293-9077-9e66da55d81f admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:57.905259 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-61093fa6-43bf-439f-aba3-92239ccc5606 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:57.906391 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-61093fa6-43bf-439f-aba3-92239ccc5606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:57.916168 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213c3856-ac99-4696-9d59-efe151e5a433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:57.918005 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213c3856-ac99-4696-9d59-efe151e5a433 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:57.920829 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213c3856-ac99-4696-9d59-efe151e5a433 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:57.921107 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213c3856-ac99-4696-9d59-efe151e5a433 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:57.921818 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213c3856-ac99-4696-9d59-efe151e5a433 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:57.925000 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213c3856-ac99-4696-9d59-efe151e5a433 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:57.931895 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1011/2032] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:57 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:57.933921 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61093fa6-43bf-439f-aba3-92239ccc5606 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:57.952044 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61093fa6-43bf-439f-aba3-92239ccc5606 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:57.967290 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fe9966-262f-407e-82ef-6d624fb4559a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:57.969043 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fe9966-262f-407e-82ef-6d624fb4559a admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:57.969043 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fe9966-262f-407e-82ef-6d624fb4559a admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:57.969043 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fe9966-262f-407e-82ef-6d624fb4559a admin admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:57.969364 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fe9966-262f-407e-82ef-6d624fb4559a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:31:57.969543 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fe9966-262f-407e-82ef-6d624fb4559a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:57.971679 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1022/2033] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:31:57 2022] GET /identity/v3/roles => generated 1427 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 17:31:57.971809 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fe9966-262f-407e-82ef-6d624fb4559a admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:57.976556 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213c3856-ac99-4696-9d59-efe151e5a433 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:57.988830 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213c3856-ac99-4696-9d59-efe151e5a433 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:57.995077 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3942f03-aa89-4c4b-94d9-9740600cbecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:57.996981 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3942f03-aa89-4c4b-94d9-9740600cbecd admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:57.997743 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3942f03-aa89-4c4b-94d9-9740600cbecd admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:57.998203 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3942f03-aa89-4c4b-94d9-9740600cbecd admin admin] PATH_INFO: `/v3/projects/e810acd39e804c379756ee1c9b2ae995/users/ac66fa5f10024cb8b0f0a093856f9dc9/roles/36577a6f46d642d4b81805ede9872490` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:57.999664 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1012/2034] 10.222.0.102 () {58 vars in 1158 bytes} [Thu Sep 1 17:31:57 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:31:58.003995 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fe9966-262f-407e-82ef-6d624fb4559a admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:58.027098 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fe9966-262f-407e-82ef-6d624fb4559a admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:58.031430 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3942f03-aa89-4c4b-94d9-9740600cbecd admin admin] RBAC: Authorizing `identity:create_grant(project_id=e810acd39e804c379756ee1c9b2ae995, user_id=ac66fa5f10024cb8b0f0a093856f9dc9, role_id=36577a6f46d642d4b81805ede9872490)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:58.043126 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3942f03-aa89-4c4b-94d9-9740600cbecd admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:58.048347 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe342ad5-e7f4-45e4-801b-7074ae26f7b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:58.048972 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe342ad5-e7f4-45e4-801b-7074ae26f7b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:58.049254 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe342ad5-e7f4-45e4-801b-7074ae26f7b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:58.049592 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe342ad5-e7f4-45e4-801b-7074ae26f7b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:58.050297 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe342ad5-e7f4-45e4-801b-7074ae26f7b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:58.052464 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe342ad5-e7f4-45e4-801b-7074ae26f7b7 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:58.077769 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1023/2035] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:31:57 2022] PUT /identity/v3/projects/e810acd39e804c379756ee1c9b2ae995/users/ac66fa5f10024cb8b0f0a093856f9dc9/roles/36577a6f46d642d4b81805ede9872490 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:58.078923 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3942f03-aa89-4c4b-94d9-9740600cbecd admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:58.114506 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1013/2036] 10.222.0.102 () {60 vars in 1158 bytes} [Thu Sep 1 17:31:58 2022] POST /identity/v3/projects => generated 382 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:31:58.122660 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe342ad5-e7f4-45e4-801b-7074ae26f7b7 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:58.236333 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3942f03-aa89-4c4b-94d9-9740600cbecd admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:58.301260 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:58.353504 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe342ad5-e7f4-45e4-801b-7074ae26f7b7 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:58.361662 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25e6320-1e71-4847-a5a8-d5ac3a3ebb70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:58.362434 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25e6320-1e71-4847-a5a8-d5ac3a3ebb70 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:58.362910 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25e6320-1e71-4847-a5a8-d5ac3a3ebb70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:58.363230 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25e6320-1e71-4847-a5a8-d5ac3a3ebb70 admin admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:58.363940 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25e6320-1e71-4847-a5a8-d5ac3a3ebb70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:58.366877 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25e6320-1e71-4847-a5a8-d5ac3a3ebb70 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:58.395448 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9486a8-4c43-4dc2-a1ea-7780e713cf1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:58.396317 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9486a8-4c43-4dc2-a1ea-7780e713cf1e admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:58.396951 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9486a8-4c43-4dc2-a1ea-7780e713cf1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:58.397343 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9486a8-4c43-4dc2-a1ea-7780e713cf1e admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:58.398493 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9486a8-4c43-4dc2-a1ea-7780e713cf1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:58.400443 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9486a8-4c43-4dc2-a1ea-7780e713cf1e admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:58.403947 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1014/2037] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:31:58 2022] POST /identity/v3/users => generated 292 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:31:58.407143 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25e6320-1e71-4847-a5a8-d5ac3a3ebb70 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:58.421688 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1024/2038] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:31:58 2022] GET /identity/v3/roles => generated 1427 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:31:58.424576 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25e6320-1e71-4847-a5a8-d5ac3a3ebb70 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:58.430864 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb47d08-6636-4fd5-a717-505330fda417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:58.432227 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb47d08-6636-4fd5-a717-505330fda417 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:58.433207 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb47d08-6636-4fd5-a717-505330fda417 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:58.434036 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb47d08-6636-4fd5-a717-505330fda417 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:58.435140 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9486a8-4c43-4dc2-a1ea-7780e713cf1e admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:58.435713 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb47d08-6636-4fd5-a717-505330fda417 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:58.438563 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb47d08-6636-4fd5-a717-505330fda417 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:58.455654 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1015/2039] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:31:58 2022] GET /identity/v3/roles => generated 1427 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:31:58.459876 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb47d08-6636-4fd5-a717-505330fda417 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:58.461041 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9486a8-4c43-4dc2-a1ea-7780e713cf1e admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:58.473458 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683e51b0-d263-43fe-a6fc-8910744522c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:58.474389 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683e51b0-d263-43fe-a6fc-8910744522c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:58.475007 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683e51b0-d263-43fe-a6fc-8910744522c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:58.475539 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683e51b0-d263-43fe-a6fc-8910744522c6 admin admin] PATH_INFO: `/v3/projects/e810acd39e804c379756ee1c9b2ae995/users/ac66fa5f10024cb8b0f0a093856f9dc9/roles/36577a6f46d642d4b81805ede9872490` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:58.479872 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb47d08-6636-4fd5-a717-505330fda417 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:58.480833 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683e51b0-d263-43fe-a6fc-8910744522c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e810acd39e804c379756ee1c9b2ae995, user_id=ac66fa5f10024cb8b0f0a093856f9dc9, role_id=36577a6f46d642d4b81805ede9872490)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:58.486593 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a3c144-2472-4cc9-837a-b2085874997d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:58.487206 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a3c144-2472-4cc9-837a-b2085874997d admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:58.487488 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a3c144-2472-4cc9-837a-b2085874997d admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:58.487760 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a3c144-2472-4cc9-837a-b2085874997d admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:58.488451 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a3c144-2472-4cc9-837a-b2085874997d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:58.489298 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683e51b0-d263-43fe-a6fc-8910744522c6 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:58.490403 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a3c144-2472-4cc9-837a-b2085874997d admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:58.507130 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1016/2040] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:31:58 2022] GET /identity/v3/roles => generated 1427 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:31:58.514164 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a3c144-2472-4cc9-837a-b2085874997d admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:58.516548 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-71a3c144-2472-4cc9-837a-b2085874997d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:58.528076 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1025/2041] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:31:58 2022] PUT /identity/v3/projects/e810acd39e804c379756ee1c9b2ae995/users/ac66fa5f10024cb8b0f0a093856f9dc9/roles/36577a6f46d642d4b81805ede9872490 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:58.528076 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683e51b0-d263-43fe-a6fc-8910744522c6 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:58.578460 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a3c144-2472-4cc9-837a-b2085874997d admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:58.645207 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683e51b0-d263-43fe-a6fc-8910744522c6 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:58.652387 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e075e87d-135a-4f40-b711-1ab6f47c27a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:58.653099 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e075e87d-135a-4f40-b711-1ab6f47c27a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:58.653409 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e075e87d-135a-4f40-b711-1ab6f47c27a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:58.653688 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e075e87d-135a-4f40-b711-1ab6f47c27a4 admin admin] PATH_INFO: `/v3/projects/5a52942c5d0c4a28a1d13aec1b33840b/users/d20088a765284255aaddf226683955b7/roles/b75cb360925e4ba5a65f14965c103ffd` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:58.669930 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e075e87d-135a-4f40-b711-1ab6f47c27a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a52942c5d0c4a28a1d13aec1b33840b, user_id=d20088a765284255aaddf226683955b7, role_id=b75cb360925e4ba5a65f14965c103ffd)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:58.672605 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e075e87d-135a-4f40-b711-1ab6f47c27a4 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:58.673635 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77396a09-3da9-44f8-aea5-3062e4c79629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:58.674454 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77396a09-3da9-44f8-aea5-3062e4c79629 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:58.675234 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77396a09-3da9-44f8-aea5-3062e4c79629 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:58.675234 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77396a09-3da9-44f8-aea5-3062e4c79629 admin admin] PATH_INFO: `/v3/projects/349c2a93a2b6418da6081ebd3042cb96` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:58.676485 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77396a09-3da9-44f8-aea5-3062e4c79629 admin admin] RBAC: Authorizing `identity:delete_project(project_id=349c2a93a2b6418da6081ebd3042cb96)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:58.678602 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77396a09-3da9-44f8-aea5-3062e4c79629 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:58.690297 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1026/2042] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:31:58 2022] PUT /identity/v3/projects/5a52942c5d0c4a28a1d13aec1b33840b/users/d20088a765284255aaddf226683955b7/roles/b75cb360925e4ba5a65f14965c103ffd => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:58.691563 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310215d7-caa7-4bea-b418-88e235ca0dff None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:58.691854 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310215d7-caa7-4bea-b418-88e235ca0dff None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:58.692157 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310215d7-caa7-4bea-b418-88e235ca0dff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:58.703077 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:58.723736 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-310215d7-caa7-4bea-b418-88e235ca0dff None None] MFA Rules not processed for user `ac66fa5f10024cb8b0f0a093856f9dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:58.725132 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-310215d7-caa7-4bea-b418-88e235ca0dff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:58.743770 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-77396a09-3da9-44f8-aea5-3062e4c79629 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '349c2a93a2b6418da6081ebd3042cb96', 'request_id': 'req-77396a09-3da9-44f8-aea5-3062e4c79629'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:58.751047 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-77396a09-3da9-44f8-aea5-3062e4c79629 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '349c2a93a2b6418da6081ebd3042cb96', 'request_id': 'req-77396a09-3da9-44f8-aea5-3062e4c79629'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:58.759252 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-77396a09-3da9-44f8-aea5-3062e4c79629 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '349c2a93a2b6418da6081ebd3042cb96', 'request_id': 'req-77396a09-3da9-44f8-aea5-3062e4c79629'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:58.773794 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-77396a09-3da9-44f8-aea5-3062e4c79629 admin admin] The token cache is being invalidate because project 349c2a93a2b6418da6081ebd3042cb96 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:31:58.774089 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-77396a09-3da9-44f8-aea5-3062e4c79629 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:58.774778 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-77396a09-3da9-44f8-aea5-3062e4c79629 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:58.777124 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1017/2043] 10.222.0.102 () {58 vars in 1238 bytes} [Thu Sep 1 17:31:58 2022] DELETE /identity/v3/projects/349c2a93a2b6418da6081ebd3042cb96 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:58.777238 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77396a09-3da9-44f8-aea5-3062e4c79629 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:58.778548 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-77396a09-3da9-44f8-aea5-3062e4c79629 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:58.825871 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1027/2044] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:58 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:58.827479 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464dfffc-1614-459c-b355-d77ea8d9583b None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:58.827966 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464dfffc-1614-459c-b355-d77ea8d9583b None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:58.828301 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464dfffc-1614-459c-b355-d77ea8d9583b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:58.862900 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-464dfffc-1614-459c-b355-d77ea8d9583b None None] MFA Rules not processed for user `d20088a765284255aaddf226683955b7`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:58.864396 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-464dfffc-1614-459c-b355-d77ea8d9583b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:58.870790 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77396a09-3da9-44f8-aea5-3062e4c79629 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:58.876292 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:58.884192 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76813d93-a80f-4987-98e3-11ebb126b7d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:58.885142 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76813d93-a80f-4987-98e3-11ebb126b7d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:58.885600 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76813d93-a80f-4987-98e3-11ebb126b7d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:58.886122 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76813d93-a80f-4987-98e3-11ebb126b7d1 admin admin] PATH_INFO: `/v3/users/71fff485c290405ab1c987a546e76a1c` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:58.887628 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76813d93-a80f-4987-98e3-11ebb126b7d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=71fff485c290405ab1c987a546e76a1c)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:58.890794 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76813d93-a80f-4987-98e3-11ebb126b7d1 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:58.944953 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1028/2045] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:58 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:58.948631 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf106749-18f1-4df8-8266-668313fdc61c None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:58.948631 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf106749-18f1-4df8-8266-668313fdc61c None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:58.948631 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf106749-18f1-4df8-8266-668313fdc61c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:58.970871 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-76813d93-a80f-4987-98e3-11ebb126b7d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71fff485c290405ab1c987a546e76a1c'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:58.972439 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-76813d93-a80f-4987-98e3-11ebb126b7d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71fff485c290405ab1c987a546e76a1c'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:58.975600 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-cf106749-18f1-4df8-8266-668313fdc61c None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:58.977258 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-cf106749-18f1-4df8-8266-668313fdc61c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:58.989485 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-76813d93-a80f-4987-98e3-11ebb126b7d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71fff485c290405ab1c987a546e76a1c'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:58.995884 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1029/2046] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:58 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:58.995884 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf106749-18f1-4df8-8266-668313fdc61c None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:58.995884 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-cf106749-18f1-4df8-8266-668313fdc61c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:59.013290 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-76813d93-a80f-4987-98e3-11ebb126b7d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71fff485c290405ab1c987a546e76a1c'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:59.015681 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-76813d93-a80f-4987-98e3-11ebb126b7d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71fff485c290405ab1c987a546e76a1c'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:31:59.034556 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1018/2047] 10.222.0.102 () {58 vars in 1229 bytes} [Thu Sep 1 17:31:58 2022] DELETE /identity/v3/users/71fff485c290405ab1c987a546e76a1c => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:59.035964 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76813d93-a80f-4987-98e3-11ebb126b7d1 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:59.045525 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-76813d93-a80f-4987-98e3-11ebb126b7d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:59.077663 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:59.083879 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf106749-18f1-4df8-8266-668313fdc61c None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:59.147648 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76813d93-a80f-4987-98e3-11ebb126b7d1 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:59.156206 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cac2db-69ae-4821-93ac-0512d4740fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:59.157070 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cac2db-69ae-4821-93ac-0512d4740fb9 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:59.157457 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cac2db-69ae-4821-93ac-0512d4740fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:59.157919 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cac2db-69ae-4821-93ac-0512d4740fb9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:59.158677 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cac2db-69ae-4821-93ac-0512d4740fb9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:59.161558 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cac2db-69ae-4821-93ac-0512d4740fb9 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:59.180064 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c885d2cd-36a7-468f-987d-5ca59f1535e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '523ed589872a45a09b6a7675a7fb0b75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:59.180715 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c885d2cd-36a7-468f-987d-5ca59f1535e8 service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:59.181517 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c885d2cd-36a7-468f-987d-5ca59f1535e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:59.181792 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c885d2cd-36a7-468f-987d-5ca59f1535e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:59.185930 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-c885d2cd-36a7-468f-987d-5ca59f1535e8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:59.203258 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1019/2048] 10.222.0.102 () {60 vars in 1158 bytes} [Thu Sep 1 17:31:59 2022] POST /identity/v3/projects => generated 382 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:31:59.204741 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4edb73-ec51-4bc2-a8f8-1bdf1c7305ed None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:59.205255 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4edb73-ec51-4bc2-a8f8-1bdf1c7305ed None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:59.205653 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4edb73-ec51-4bc2-a8f8-1bdf1c7305ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:59.241745 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-6e4edb73-ec51-4bc2-a8f8-1bdf1c7305ed None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:59.243871 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-6e4edb73-ec51-4bc2-a8f8-1bdf1c7305ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:59.276608 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c885d2cd-36a7-468f-987d-5ca59f1535e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:59.278628 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c885d2cd-36a7-468f-987d-5ca59f1535e8 service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:59.292797 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1030/2049] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:31:58 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:31:59.295516 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e7c69c-4b07-4e4b-b098-33e38030e7ba None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:59.295804 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e7c69c-4b07-4e4b-b098-33e38030e7ba None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:59.296466 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e7c69c-4b07-4e4b-b098-33e38030e7ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:59.327790 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-e5e7c69c-4b07-4e4b-b098-33e38030e7ba None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:59.329445 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-e5e7c69c-4b07-4e4b-b098-33e38030e7ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:59.347149 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:59.362342 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1020/2050] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:59 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:59.368257 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4edb73-ec51-4bc2-a8f8-1bdf1c7305ed None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:59.389197 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4edb73-ec51-4bc2-a8f8-1bdf1c7305ed None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:59.402268 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43fafa5-26f3-4275-b5d2-e959ec74dc84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:59.403670 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43fafa5-26f3-4275-b5d2-e959ec74dc84 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:59.406412 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43fafa5-26f3-4275-b5d2-e959ec74dc84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:59.406787 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43fafa5-26f3-4275-b5d2-e959ec74dc84 admin admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:59.408671 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43fafa5-26f3-4275-b5d2-e959ec74dc84 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:59.412074 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43fafa5-26f3-4275-b5d2-e959ec74dc84 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:59.419953 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1031/2051] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:31:59 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:59.424711 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e7c69c-4b07-4e4b-b098-33e38030e7ba None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:59.450282 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e7c69c-4b07-4e4b-b098-33e38030e7ba None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:59.463976 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25280440-14b8-4456-8ed6-630a27c6838a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:59.468467 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25280440-14b8-4456-8ed6-630a27c6838a admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:59.468832 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25280440-14b8-4456-8ed6-630a27c6838a admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:59.469918 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25280440-14b8-4456-8ed6-630a27c6838a admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:59.469918 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25280440-14b8-4456-8ed6-630a27c6838a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:31:59.473718 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25280440-14b8-4456-8ed6-630a27c6838a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:59.473841 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1021/2052] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:31:59 2022] POST /identity/v3/users => generated 298 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:31:59.473841 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43fafa5-26f3-4275-b5d2-e959ec74dc84 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:59.474306 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25280440-14b8-4456-8ed6-630a27c6838a admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:59.482134 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-a43fafa5-26f3-4275-b5d2-e959ec74dc84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:59.503474 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1032/2053] 10.222.0.102 () {58 vars in 1158 bytes} [Thu Sep 1 17:31:59 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:31:59.505640 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25280440-14b8-4456-8ed6-630a27c6838a admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:59.527033 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25280440-14b8-4456-8ed6-630a27c6838a admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:59.534060 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13baa5b7-37cc-4c17-a58e-f3c7ee1e22eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:59.535489 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13baa5b7-37cc-4c17-a58e-f3c7ee1e22eb admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:59.535958 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13baa5b7-37cc-4c17-a58e-f3c7ee1e22eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:59.536346 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13baa5b7-37cc-4c17-a58e-f3c7ee1e22eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:59.537376 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13baa5b7-37cc-4c17-a58e-f3c7ee1e22eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:59.539670 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13baa5b7-37cc-4c17-a58e-f3c7ee1e22eb admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:59.557840 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1033/2054] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:31:59 2022] GET /identity/v3/roles => generated 1427 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:31:59.563511 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13baa5b7-37cc-4c17-a58e-f3c7ee1e22eb admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:59.579908 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13baa5b7-37cc-4c17-a58e-f3c7ee1e22eb admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:59.585414 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3d01f9-3af4-4f32-84fa-dea4a0bdd5ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:59.586823 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3d01f9-3af4-4f32-84fa-dea4a0bdd5ed admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:59.587107 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3d01f9-3af4-4f32-84fa-dea4a0bdd5ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:59.587378 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3d01f9-3af4-4f32-84fa-dea4a0bdd5ed admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:59.591197 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3d01f9-3af4-4f32-84fa-dea4a0bdd5ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:59.593207 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3d01f9-3af4-4f32-84fa-dea4a0bdd5ed admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:59.597452 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43fafa5-26f3-4275-b5d2-e959ec74dc84 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:59.609402 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8ad1c7-be0f-417a-a282-ce9d7d712ac7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:59.610201 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8ad1c7-be0f-417a-a282-ce9d7d712ac7 service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:59.610486 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8ad1c7-be0f-417a-a282-ce9d7d712ac7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:59.610760 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8ad1c7-be0f-417a-a282-ce9d7d712ac7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:59.631753 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1034/2055] 10.222.0.102 () {60 vars in 1158 bytes} [Thu Sep 1 17:31:59 2022] POST /identity/v3/projects => generated 386 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:31:59.631979 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8ad1c7-be0f-417a-a282-ce9d7d712ac7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:59.633958 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8ad1c7-be0f-417a-a282-ce9d7d712ac7 service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:59.634721 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3d01f9-3af4-4f32-84fa-dea4a0bdd5ed admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:59.664648 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1022/2056] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:31:59 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:31:59.669053 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8ad1c7-be0f-417a-a282-ce9d7d712ac7 service neutron] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:59.694176 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8ad1c7-be0f-417a-a282-ce9d7d712ac7 service neutron] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:59.770400 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:59.795095 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3d01f9-3af4-4f32-84fa-dea4a0bdd5ed admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:59.805457 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cff071-4795-4887-a52b-29d99b18d083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:59.806207 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cff071-4795-4887-a52b-29d99b18d083 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:59.806734 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cff071-4795-4887-a52b-29d99b18d083 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:59.806848 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cff071-4795-4887-a52b-29d99b18d083 admin admin] PATH_INFO: `/v3/projects/0c93afe3b7c84ce28a4b0f5f9f3dbf8f/users/4ddc2f14be0a4b7e96102a7ab76e6cbe/roles/36577a6f46d642d4b81805ede9872490` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:59.820963 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f1b8b5-4b1f-458b-a46a-163c692fc788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:59.831451 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f1b8b5-4b1f-458b-a46a-163c692fc788 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:59.831746 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f1b8b5-4b1f-458b-a46a-163c692fc788 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:59.833232 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f1b8b5-4b1f-458b-a46a-163c692fc788 admin admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:59.835071 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f1b8b5-4b1f-458b-a46a-163c692fc788 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:59.839346 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f1b8b5-4b1f-458b-a46a-163c692fc788 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:59.841535 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cff071-4795-4887-a52b-29d99b18d083 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c93afe3b7c84ce28a4b0f5f9f3dbf8f, user_id=4ddc2f14be0a4b7e96102a7ab76e6cbe, role_id=36577a6f46d642d4b81805ede9872490)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:59.845594 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cff071-4795-4887-a52b-29d99b18d083 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:59.892240 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1035/2057] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:31:59 2022] PUT /identity/v3/projects/0c93afe3b7c84ce28a4b0f5f9f3dbf8f/users/4ddc2f14be0a4b7e96102a7ab76e6cbe/roles/36577a6f46d642d4b81805ede9872490 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:31:59.892240 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5c9ac6-aac3-4d4d-ae6f-1aeeb8a92231 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:59.892240 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5c9ac6-aac3-4d4d-ae6f-1aeeb8a92231 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:59.892240 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5c9ac6-aac3-4d4d-ae6f-1aeeb8a92231 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:59.892974 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1023/2058] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:31:59 2022] POST /identity/v3/users => generated 294 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:31:59.893573 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1036/2059] 10.222.0.102 () {54 vars in 926 bytes} [Thu Sep 1 17:31:59 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:31:59.894373 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f1b8b5-4b1f-458b-a46a-163c692fc788 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:59.896072 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-a7f1b8b5-4b1f-458b-a46a-163c692fc788 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:59.899962 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5c9ac6-aac3-4d4d-ae6f-1aeeb8a92231 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:59.911269 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f1b8b5-4b1f-458b-a46a-163c692fc788 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:59.996321 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c040526-8d1e-450a-b348-1507ce600d52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:59.997633 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:59.998731 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c040526-8d1e-450a-b348-1507ce600d52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:59.999241 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c040526-8d1e-450a-b348-1507ce600d52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:59.999705 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c040526-8d1e-450a-b348-1507ce600d52 admin admin] PATH_INFO: `/v3/users/1c2839729b5c4814b80113e9df32aade` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:00.001567 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c040526-8d1e-450a-b348-1507ce600d52 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c2839729b5c4814b80113e9df32aade)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:00.004753 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c040526-8d1e-450a-b348-1507ce600d52 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:00.017875 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5c9ac6-aac3-4d4d-ae6f-1aeeb8a92231 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:00.023181 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551378cd-b1bf-45f5-84f9-b474b49ddcfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:00.023862 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551378cd-b1bf-45f5-84f9-b474b49ddcfc admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:00.024146 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551378cd-b1bf-45f5-84f9-b474b49ddcfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:00.024419 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551378cd-b1bf-45f5-84f9-b474b49ddcfc admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:00.025565 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551378cd-b1bf-45f5-84f9-b474b49ddcfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:00.027484 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551378cd-b1bf-45f5-84f9-b474b49ddcfc admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:00.044646 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1037/2060] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:31:59 2022] GET /identity/v3/roles => generated 1427 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:32:00.044646 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551378cd-b1bf-45f5-84f9-b474b49ddcfc admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:00.054748 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551378cd-b1bf-45f5-84f9-b474b49ddcfc admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:00.060296 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989842ec-563b-4f97-a4fa-a5459c612678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:00.060902 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989842ec-563b-4f97-a4fa-a5459c612678 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:00.061184 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989842ec-563b-4f97-a4fa-a5459c612678 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:00.061508 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989842ec-563b-4f97-a4fa-a5459c612678 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:00.062149 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989842ec-563b-4f97-a4fa-a5459c612678 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:00.063821 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989842ec-563b-4f97-a4fa-a5459c612678 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:00.073459 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-2c040526-8d1e-450a-b348-1507ce600d52 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c2839729b5c4814b80113e9df32aade'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:00.074395 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1038/2061] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:32:00 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:32:00.074792 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-2c040526-8d1e-450a-b348-1507ce600d52 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c2839729b5c4814b80113e9df32aade'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:00.079203 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b46c21-94ed-419f-9c48-c4994a827766 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:00.079497 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b46c21-94ed-419f-9c48-c4994a827766 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:00.079775 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b46c21-94ed-419f-9c48-c4994a827766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:00.092557 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-2c040526-8d1e-450a-b348-1507ce600d52 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c2839729b5c4814b80113e9df32aade'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:00.102153 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-99b46c21-94ed-419f-9c48-c4994a827766 None None] MFA Rules not processed for user `5f8293316baf4774b62fc11d55d50865`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:32:00.103472 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-99b46c21-94ed-419f-9c48-c4994a827766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:00.111980 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-2c040526-8d1e-450a-b348-1507ce600d52 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c2839729b5c4814b80113e9df32aade'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:00.112763 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-2c040526-8d1e-450a-b348-1507ce600d52 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c2839729b5c4814b80113e9df32aade'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:00.132487 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1024/2062] 10.222.0.102 () {58 vars in 1229 bytes} [Thu Sep 1 17:31:59 2022] DELETE /identity/v3/users/1c2839729b5c4814b80113e9df32aade => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:32:00.134498 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c040526-8d1e-450a-b348-1507ce600d52 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:00.138718 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-2c040526-8d1e-450a-b348-1507ce600d52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:00.199473 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:32:00.214058 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1039/2063] 10.222.0.102 () {58 vars in 1023 bytes} [Thu Sep 1 17:32:00 2022] POST /identity/v3/auth/tokens => generated 3382 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:32:00.215416 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b46c21-94ed-419f-9c48-c4994a827766 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:00.217063 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-99b46c21-94ed-419f-9c48-c4994a827766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:00.225502 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c040526-8d1e-450a-b348-1507ce600d52 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:00.231442 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a839c51-067c-4a1b-92e6-19c3826e9ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:00.232306 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a839c51-067c-4a1b-92e6-19c3826e9ade admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:00.232736 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a839c51-067c-4a1b-92e6-19c3826e9ade admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:00.233170 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a839c51-067c-4a1b-92e6-19c3826e9ade admin admin] PATH_INFO: `/v3/projects/0c93afe3b7c84ce28a4b0f5f9f3dbf8f/users/4ddc2f14be0a4b7e96102a7ab76e6cbe/roles/36577a6f46d642d4b81805ede9872490` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:00.233967 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b46c21-94ed-419f-9c48-c4994a827766 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:00.235756 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a839c51-067c-4a1b-92e6-19c3826e9ade admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c93afe3b7c84ce28a4b0f5f9f3dbf8f, user_id=4ddc2f14be0a4b7e96102a7ab76e6cbe, role_id=36577a6f46d642d4b81805ede9872490)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:00.238943 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7aa50b-7eb3-4be1-b5de-6c356f308844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:00.239444 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a839c51-067c-4a1b-92e6-19c3826e9ade admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:00.239562 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7aa50b-7eb3-4be1-b5de-6c356f308844 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:00.239808 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7aa50b-7eb3-4be1-b5de-6c356f308844 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:00.240078 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7aa50b-7eb3-4be1-b5de-6c356f308844 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:00.240722 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7aa50b-7eb3-4be1-b5de-6c356f308844 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:00.242380 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7aa50b-7eb3-4be1-b5de-6c356f308844 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:00.253868 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1025/2064] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:32:00 2022] PUT /identity/v3/projects/0c93afe3b7c84ce28a4b0f5f9f3dbf8f/users/4ddc2f14be0a4b7e96102a7ab76e6cbe/roles/36577a6f46d642d4b81805ede9872490 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:32:00.254167 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1040/2065] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:32:00 2022] GET /identity/v3/roles => generated 1427 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:32:00.255164 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a839c51-067c-4a1b-92e6-19c3826e9ade admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:00.255386 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c464dbe-e034-454a-a95e-f938fae04905 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:00.255671 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c464dbe-e034-454a-a95e-f938fae04905 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:00.255946 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c464dbe-e034-454a-a95e-f938fae04905 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:00.262416 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-7a839c51-067c-4a1b-92e6-19c3826e9ade admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:00.281533 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-0c464dbe-e034-454a-a95e-f938fae04905 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:32:00.282918 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-0c464dbe-e034-454a-a95e-f938fae04905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:00.349462 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a839c51-067c-4a1b-92e6-19c3826e9ade admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:00.357713 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107359b6-0a9f-4c61-809d-1cbea1965dbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:00.358354 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107359b6-0a9f-4c61-809d-1cbea1965dbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:00.358644 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107359b6-0a9f-4c61-809d-1cbea1965dbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:00.359122 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107359b6-0a9f-4c61-809d-1cbea1965dbf admin admin] PATH_INFO: `/v3/projects/de1af01c5acc4bbf928e937725b6e9d8` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:00.360457 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1041/2066] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:32:00 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:32:00.361584 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c464dbe-e034-454a-a95e-f938fae04905 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:00.364222 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107359b6-0a9f-4c61-809d-1cbea1965dbf admin admin] RBAC: Authorizing `identity:delete_project(project_id=de1af01c5acc4bbf928e937725b6e9d8)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:00.366401 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-0c464dbe-e034-454a-a95e-f938fae04905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:00.366672 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107359b6-0a9f-4c61-809d-1cbea1965dbf admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:00.435785 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-107359b6-0a9f-4c61-809d-1cbea1965dbf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de1af01c5acc4bbf928e937725b6e9d8', 'request_id': 'req-107359b6-0a9f-4c61-809d-1cbea1965dbf'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:00.441406 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:32:00.444554 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-107359b6-0a9f-4c61-809d-1cbea1965dbf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de1af01c5acc4bbf928e937725b6e9d8', 'request_id': 'req-107359b6-0a9f-4c61-809d-1cbea1965dbf'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:00.457289 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-107359b6-0a9f-4c61-809d-1cbea1965dbf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de1af01c5acc4bbf928e937725b6e9d8', 'request_id': 'req-107359b6-0a9f-4c61-809d-1cbea1965dbf'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:00.465805 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c464dbe-e034-454a-a95e-f938fae04905 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:00.475470 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-107359b6-0a9f-4c61-809d-1cbea1965dbf admin admin] The token cache is being invalidate because project de1af01c5acc4bbf928e937725b6e9d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:32:00.475940 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-107359b6-0a9f-4c61-809d-1cbea1965dbf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:00.476416 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-107359b6-0a9f-4c61-809d-1cbea1965dbf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:00.477951 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1026/2067] 10.222.0.102 () {58 vars in 1238 bytes} [Thu Sep 1 17:32:00 2022] DELETE /identity/v3/projects/de1af01c5acc4bbf928e937725b6e9d8 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:32:00.482544 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d6d84d-c51f-49fb-a109-0f4ae2d936ec None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:00.482832 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d6d84d-c51f-49fb-a109-0f4ae2d936ec None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:00.483109 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d6d84d-c51f-49fb-a109-0f4ae2d936ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:00.513352 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-a0d6d84d-c51f-49fb-a109-0f4ae2d936ec None None] MFA Rules not processed for user `4ddc2f14be0a4b7e96102a7ab76e6cbe`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:32:00.514561 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-a0d6d84d-c51f-49fb-a109-0f4ae2d936ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:00.545145 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a18e4f0-b878-4dd5-9009-12e7db2faee7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f8293316baf4774b62fc11d55d50865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:00.545789 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a18e4f0-b878-4dd5-9009-12e7db2faee7 service nova] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:00.546071 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a18e4f0-b878-4dd5-9009-12e7db2faee7 service nova] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:00.546349 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a18e4f0-b878-4dd5-9009-12e7db2faee7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:00.548125 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-8a18e4f0-b878-4dd5-9009-12e7db2faee7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:00.560457 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a18e4f0-b878-4dd5-9009-12e7db2faee7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:00.562090 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a18e4f0-b878-4dd5-9009-12e7db2faee7 service nova] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:00.565497 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:32:00.574171 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1042/2068] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:32:00 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:32:00.575652 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a18e4f0-b878-4dd5-9009-12e7db2faee7 service nova] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:00.577782 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-8a18e4f0-b878-4dd5-9009-12e7db2faee7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:00.584835 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1027/2069] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:32:00 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:32:00.585722 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d6d84d-c51f-49fb-a109-0f4ae2d936ec None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:00.586974 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-a0d6d84d-c51f-49fb-a109-0f4ae2d936ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:00.652378 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a18e4f0-b878-4dd5-9009-12e7db2faee7 service nova] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:00.657757 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75712c41-dc4e-46e4-bd15-9b15eef72077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:00.658420 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75712c41-dc4e-46e4-bd15-9b15eef72077 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:00.658712 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75712c41-dc4e-46e4-bd15-9b15eef72077 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:00.658988 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75712c41-dc4e-46e4-bd15-9b15eef72077 admin admin] PATH_INFO: `/v3/projects/7514891da2db401cbd983bfc04b8da6c/users/fb82355daa6a4346ab93fb058788c1fb/roles/b75cb360925e4ba5a65f14965c103ffd` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:00.659090 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d6d84d-c51f-49fb-a109-0f4ae2d936ec None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:00.663099 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dc8c8f-c0ad-4674-a43d-840dc2a3e7d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:00.663875 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dc8c8f-c0ad-4674-a43d-840dc2a3e7d9 service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:00.664159 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dc8c8f-c0ad-4674-a43d-840dc2a3e7d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:00.664560 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dc8c8f-c0ad-4674-a43d-840dc2a3e7d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:00.666437 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-f5dc8c8f-c0ad-4674-a43d-840dc2a3e7d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:00.676269 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75712c41-dc4e-46e4-bd15-9b15eef72077 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7514891da2db401cbd983bfc04b8da6c, user_id=fb82355daa6a4346ab93fb058788c1fb, role_id=b75cb360925e4ba5a65f14965c103ffd)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:00.678981 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75712c41-dc4e-46e4-bd15-9b15eef72077 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:00.679559 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dc8c8f-c0ad-4674-a43d-840dc2a3e7d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:00.680998 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dc8c8f-c0ad-4674-a43d-840dc2a3e7d9 service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:00.691863 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1028/2070] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:32:00 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:32:00.692508 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dc8c8f-c0ad-4674-a43d-840dc2a3e7d9 service neutron] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:00.694647 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-f5dc8c8f-c0ad-4674-a43d-840dc2a3e7d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:00.697403 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1043/2071] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:32:00 2022] PUT /identity/v3/projects/7514891da2db401cbd983bfc04b8da6c/users/fb82355daa6a4346ab93fb058788c1fb/roles/b75cb360925e4ba5a65f14965c103ffd => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:32:00.698505 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75712c41-dc4e-46e4-bd15-9b15eef72077 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:00.700452 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-75712c41-dc4e-46e4-bd15-9b15eef72077 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:00.708936 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dc8c8f-c0ad-4674-a43d-840dc2a3e7d9 service neutron] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:00.770235 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75712c41-dc4e-46e4-bd15-9b15eef72077 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:00.774542 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0781b10-bbcf-455a-899d-513f152f41e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '523ed589872a45a09b6a7675a7fb0b75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:00.775371 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0781b10-bbcf-455a-899d-513f152f41e4 service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:00.775715 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0781b10-bbcf-455a-899d-513f152f41e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:00.776091 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0781b10-bbcf-455a-899d-513f152f41e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:00.777599 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d5af31-58c2-43ef-8880-698c7050df2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:00.778811 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d5af31-58c2-43ef-8880-698c7050df2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:00.779119 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d5af31-58c2-43ef-8880-698c7050df2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:00.779348 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d5af31-58c2-43ef-8880-698c7050df2f admin admin] PATH_INFO: `/v3/users/9b16b9e35cc740febdc39f6561d65cb1` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:00.780515 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d5af31-58c2-43ef-8880-698c7050df2f admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b16b9e35cc740febdc39f6561d65cb1)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:00.782467 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d5af31-58c2-43ef-8880-698c7050df2f admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:00.806504 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:32:00.843834 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-87d5af31-58c2-43ef-8880-698c7050df2f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b16b9e35cc740febdc39f6561d65cb1'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:00.844792 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-87d5af31-58c2-43ef-8880-698c7050df2f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b16b9e35cc740febdc39f6561d65cb1'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:00.864980 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-87d5af31-58c2-43ef-8880-698c7050df2f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b16b9e35cc740febdc39f6561d65cb1'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:00.874981 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-87d5af31-58c2-43ef-8880-698c7050df2f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b16b9e35cc740febdc39f6561d65cb1'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:00.875748 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-87d5af31-58c2-43ef-8880-698c7050df2f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b16b9e35cc740febdc39f6561d65cb1'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:00.876276 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0781b10-bbcf-455a-899d-513f152f41e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:00.878017 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0781b10-bbcf-455a-899d-513f152f41e4 service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:00.889919 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1044/2072] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:32:00 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:32:00.891282 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4971ee73-9aa1-433b-8b3e-443f561bcb86 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:00.891479 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4971ee73-9aa1-433b-8b3e-443f561bcb86 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:00.891756 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4971ee73-9aa1-433b-8b3e-443f561bcb86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:00.893551 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1029/2073] 10.222.0.102 () {58 vars in 1229 bytes} [Thu Sep 1 17:32:00 2022] DELETE /identity/v3/users/9b16b9e35cc740febdc39f6561d65cb1 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:32:00.924891 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-4971ee73-9aa1-433b-8b3e-443f561bcb86 None None] MFA Rules not processed for user `fb82355daa6a4346ab93fb058788c1fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:32:00.934505 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-4971ee73-9aa1-433b-8b3e-443f561bcb86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:00.981152 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d5af31-58c2-43ef-8880-698c7050df2f admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:00.982862 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-87d5af31-58c2-43ef-8880-698c7050df2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:01.001251 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:32:01.061722 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1045/2074] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:32:00 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:32:01.064086 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d5af31-58c2-43ef-8880-698c7050df2f admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:01.068190 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4971ee73-9aa1-433b-8b3e-443f561bcb86 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:01.068951 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e81edff-6a42-4cfc-8e4d-6f384dc2d08c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '523ed589872a45a09b6a7675a7fb0b75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:01.069599 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e81edff-6a42-4cfc-8e4d-6f384dc2d08c service cinder] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:01.069709 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-4971ee73-9aa1-433b-8b3e-443f561bcb86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:01.069903 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e81edff-6a42-4cfc-8e4d-6f384dc2d08c service cinder] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:01.070147 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e81edff-6a42-4cfc-8e4d-6f384dc2d08c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:01.072422 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-5e81edff-6a42-4cfc-8e4d-6f384dc2d08c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:01.162097 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e81edff-6a42-4cfc-8e4d-6f384dc2d08c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:01.163970 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e81edff-6a42-4cfc-8e4d-6f384dc2d08c service cinder] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:01.190058 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1030/2075] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:32:00 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 17:32:01.214180 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e81edff-6a42-4cfc-8e4d-6f384dc2d08c service cinder] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:01.215675 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-5e81edff-6a42-4cfc-8e4d-6f384dc2d08c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:01.238319 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4971ee73-9aa1-433b-8b3e-443f561bcb86 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:01.238505 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e81edff-6a42-4cfc-8e4d-6f384dc2d08c service cinder] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:01.244052 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e610aa48-f1e5-484a-9f11-99d65db91259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:01.244765 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e610aa48-f1e5-484a-9f11-99d65db91259 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:01.244919 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce6d3f2-8414-4f30-8f55-fb25e7cf67e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:01.245100 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e610aa48-f1e5-484a-9f11-99d65db91259 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:01.245344 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e610aa48-f1e5-484a-9f11-99d65db91259 admin admin] PATH_INFO: `/v3/projects/98d5dfc87a214d0fbc086b9eff7a5bf2` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:01.246452 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce6d3f2-8414-4f30-8f55-fb25e7cf67e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:01.247115 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e610aa48-f1e5-484a-9f11-99d65db91259 admin admin] RBAC: Authorizing `identity:delete_project(project_id=98d5dfc87a214d0fbc086b9eff7a5bf2)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:01.247355 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce6d3f2-8414-4f30-8f55-fb25e7cf67e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:01.247701 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce6d3f2-8414-4f30-8f55-fb25e7cf67e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:01.248526 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce6d3f2-8414-4f30-8f55-fb25e7cf67e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:01.249161 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e610aa48-f1e5-484a-9f11-99d65db91259 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:01.251205 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce6d3f2-8414-4f30-8f55-fb25e7cf67e3 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:01.295740 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1046/2076] 10.222.0.102 () {60 vars in 1158 bytes} [Thu Sep 1 17:32:01 2022] POST /identity/v3/projects => generated 386 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:32:01.300950 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce6d3f2-8414-4f30-8f55-fb25e7cf67e3 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:01.319641 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-e610aa48-f1e5-484a-9f11-99d65db91259 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98d5dfc87a214d0fbc086b9eff7a5bf2', 'request_id': 'req-e610aa48-f1e5-484a-9f11-99d65db91259'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:01.326570 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-e610aa48-f1e5-484a-9f11-99d65db91259 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98d5dfc87a214d0fbc086b9eff7a5bf2', 'request_id': 'req-e610aa48-f1e5-484a-9f11-99d65db91259'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:01.336649 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-e610aa48-f1e5-484a-9f11-99d65db91259 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98d5dfc87a214d0fbc086b9eff7a5bf2', 'request_id': 'req-e610aa48-f1e5-484a-9f11-99d65db91259'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:01.364588 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:32:01.365419 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-e610aa48-f1e5-484a-9f11-99d65db91259 admin admin] The token cache is being invalidate because project 98d5dfc87a214d0fbc086b9eff7a5bf2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:32:01.365800 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-e610aa48-f1e5-484a-9f11-99d65db91259 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:01.367070 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-e610aa48-f1e5-484a-9f11-99d65db91259 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:01.369533 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1031/2077] 10.222.0.102 () {58 vars in 1238 bytes} [Thu Sep 1 17:32:01 2022] DELETE /identity/v3/projects/98d5dfc87a214d0fbc086b9eff7a5bf2 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:32:01.375264 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e610aa48-f1e5-484a-9f11-99d65db91259 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:01.376982 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-e610aa48-f1e5-484a-9f11-99d65db91259 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:01.397196 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce6d3f2-8414-4f30-8f55-fb25e7cf67e3 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:01.450942 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:32:01.473767 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e610aa48-f1e5-484a-9f11-99d65db91259 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:01.479934 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab7a1f8-d0dd-49aa-8d2f-73ff2cf320e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:01.485011 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab7a1f8-d0dd-49aa-8d2f-73ff2cf320e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:01.485345 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab7a1f8-d0dd-49aa-8d2f-73ff2cf320e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:01.485618 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab7a1f8-d0dd-49aa-8d2f-73ff2cf320e7 admin admin] PATH_INFO: `/v3/users/cfceddaf9335487f990f1a32bf18e97e` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:01.486879 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab7a1f8-d0dd-49aa-8d2f-73ff2cf320e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cfceddaf9335487f990f1a32bf18e97e)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:01.488815 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab7a1f8-d0dd-49aa-8d2f-73ff2cf320e7 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:01.495588 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897e1046-e89c-4793-b13b-e4fd73ea3227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:01.498491 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897e1046-e89c-4793-b13b-e4fd73ea3227 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:01.499167 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897e1046-e89c-4793-b13b-e4fd73ea3227 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:01.499754 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897e1046-e89c-4793-b13b-e4fd73ea3227 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:01.500617 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897e1046-e89c-4793-b13b-e4fd73ea3227 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:01.503064 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897e1046-e89c-4793-b13b-e4fd73ea3227 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:01.533370 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1047/2078] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:32:01 2022] POST /identity/v3/users => generated 294 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:32:01.538481 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897e1046-e89c-4793-b13b-e4fd73ea3227 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:01.541272 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-897e1046-e89c-4793-b13b-e4fd73ea3227 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:01.559023 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-cab7a1f8-d0dd-49aa-8d2f-73ff2cf320e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cfceddaf9335487f990f1a32bf18e97e'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:01.559416 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897e1046-e89c-4793-b13b-e4fd73ea3227 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:01.562011 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-cab7a1f8-d0dd-49aa-8d2f-73ff2cf320e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cfceddaf9335487f990f1a32bf18e97e'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:01.566135 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810708cf-4a7d-42b9-b6b0-164c75bb912c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:01.567255 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810708cf-4a7d-42b9-b6b0-164c75bb912c admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:01.567802 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810708cf-4a7d-42b9-b6b0-164c75bb912c admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:01.568385 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810708cf-4a7d-42b9-b6b0-164c75bb912c admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:01.569543 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810708cf-4a7d-42b9-b6b0-164c75bb912c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:01.573163 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810708cf-4a7d-42b9-b6b0-164c75bb912c admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:01.584407 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-cab7a1f8-d0dd-49aa-8d2f-73ff2cf320e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cfceddaf9335487f990f1a32bf18e97e'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:01.598280 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1048/2079] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:32:01 2022] GET /identity/v3/roles => generated 1427 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:32:01.602044 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-cab7a1f8-d0dd-49aa-8d2f-73ff2cf320e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cfceddaf9335487f990f1a32bf18e97e'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:01.604671 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-cab7a1f8-d0dd-49aa-8d2f-73ff2cf320e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cfceddaf9335487f990f1a32bf18e97e'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:01.608049 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810708cf-4a7d-42b9-b6b0-164c75bb912c admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:01.613490 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-810708cf-4a7d-42b9-b6b0-164c75bb912c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:01.635024 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1032/2080] 10.222.0.102 () {58 vars in 1229 bytes} [Thu Sep 1 17:32:01 2022] DELETE /identity/v3/users/cfceddaf9335487f990f1a32bf18e97e => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:32:01.639498 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab7a1f8-d0dd-49aa-8d2f-73ff2cf320e7 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:01.640776 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-cab7a1f8-d0dd-49aa-8d2f-73ff2cf320e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:01.645140 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810708cf-4a7d-42b9-b6b0-164c75bb912c admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:01.780257 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ad0e93-e8cd-40fc-bf0d-02838f37e8df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:01.780948 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ad0e93-e8cd-40fc-bf0d-02838f37e8df admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:01.781240 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ad0e93-e8cd-40fc-bf0d-02838f37e8df admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:01.781600 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ad0e93-e8cd-40fc-bf0d-02838f37e8df admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:01.782260 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ad0e93-e8cd-40fc-bf0d-02838f37e8df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:01.784200 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ad0e93-e8cd-40fc-bf0d-02838f37e8df admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:01.800859 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1049/2081] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:32:01 2022] GET /identity/v3/roles => generated 1427 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:32:01.801954 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ad0e93-e8cd-40fc-bf0d-02838f37e8df admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:01.804329 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-d4ad0e93-e8cd-40fc-bf0d-02838f37e8df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:01.806881 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab7a1f8-d0dd-49aa-8d2f-73ff2cf320e7 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:01.825652 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22dedb3-ea53-4551-b8ac-22089c400264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:01.826234 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22dedb3-ea53-4551-b8ac-22089c400264 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:01.826507 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22dedb3-ea53-4551-b8ac-22089c400264 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:01.826791 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22dedb3-ea53-4551-b8ac-22089c400264 admin admin] PATH_INFO: `/v3/projects/b31690f5ad9d4e6c97d66d8f8cddd4d8` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:01.829149 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22dedb3-ea53-4551-b8ac-22089c400264 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b31690f5ad9d4e6c97d66d8f8cddd4d8)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:01.831236 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22dedb3-ea53-4551-b8ac-22089c400264 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:01.836016 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ad0e93-e8cd-40fc-bf0d-02838f37e8df admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:01.846525 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc83a58f-1434-4e7e-94f4-f2de6f80e06b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:01.851240 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc83a58f-1434-4e7e-94f4-f2de6f80e06b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:01.851240 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc83a58f-1434-4e7e-94f4-f2de6f80e06b admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:01.851240 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc83a58f-1434-4e7e-94f4-f2de6f80e06b admin admin] PATH_INFO: `/v3/users/d20088a765284255aaddf226683955b7` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:01.854458 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc83a58f-1434-4e7e-94f4-f2de6f80e06b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d20088a765284255aaddf226683955b7)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:01.856484 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc83a58f-1434-4e7e-94f4-f2de6f80e06b admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:01.921285 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-c22dedb3-ea53-4551-b8ac-22089c400264 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b31690f5ad9d4e6c97d66d8f8cddd4d8', 'request_id': 'req-c22dedb3-ea53-4551-b8ac-22089c400264'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:01.938723 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-c22dedb3-ea53-4551-b8ac-22089c400264 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b31690f5ad9d4e6c97d66d8f8cddd4d8', 'request_id': 'req-c22dedb3-ea53-4551-b8ac-22089c400264'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:01.967749 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-c22dedb3-ea53-4551-b8ac-22089c400264 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b31690f5ad9d4e6c97d66d8f8cddd4d8', 'request_id': 'req-c22dedb3-ea53-4551-b8ac-22089c400264'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:01.968088 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-fc83a58f-1434-4e7e-94f4-f2de6f80e06b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd20088a765284255aaddf226683955b7'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:01.971647 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:32:01.979127 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:32:01.987452 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-c22dedb3-ea53-4551-b8ac-22089c400264 admin admin] The token cache is being invalidate because project b31690f5ad9d4e6c97d66d8f8cddd4d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:32:01.989728 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-c22dedb3-ea53-4551-b8ac-22089c400264 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:01.990997 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-c22dedb3-ea53-4551-b8ac-22089c400264 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:01.994501 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1033/2082] 10.222.0.102 () {58 vars in 1238 bytes} [Thu Sep 1 17:32:01 2022] DELETE /identity/v3/projects/b31690f5ad9d4e6c97d66d8f8cddd4d8 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:32:01.995558 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22dedb3-ea53-4551-b8ac-22089c400264 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:02.000954 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-c22dedb3-ea53-4551-b8ac-22089c400264 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:02.006385 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-fc83a58f-1434-4e7e-94f4-f2de6f80e06b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd20088a765284255aaddf226683955b7'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:02.024195 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-fc83a58f-1434-4e7e-94f4-f2de6f80e06b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd20088a765284255aaddf226683955b7'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:02.025082 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-fc83a58f-1434-4e7e-94f4-f2de6f80e06b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd20088a765284255aaddf226683955b7'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:02.029594 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-fc83a58f-1434-4e7e-94f4-f2de6f80e06b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd20088a765284255aaddf226683955b7'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:02.053789 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1050/2083] 10.222.0.102 () {58 vars in 1229 bytes} [Thu Sep 1 17:32:01 2022] DELETE /identity/v3/users/d20088a765284255aaddf226683955b7 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:32:02.057904 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4c9de8-5924-4598-aac7-0e4ce2e401ff None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:02.058360 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4c9de8-5924-4598-aac7-0e4ce2e401ff None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:02.058779 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4c9de8-5924-4598-aac7-0e4ce2e401ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:02.101499 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-cf4c9de8-5924-4598-aac7-0e4ce2e401ff None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:32:02.102742 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-cf4c9de8-5924-4598-aac7-0e4ce2e401ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:02.125699 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22dedb3-ea53-4551-b8ac-22089c400264 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:02.217677 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1051/2084] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:32:02 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:32:02.220378 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2b6fac-2b0e-475f-8f43-8c46357c662f None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:02.220852 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2b6fac-2b0e-475f-8f43-8c46357c662f None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:02.221604 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2b6fac-2b0e-475f-8f43-8c46357c662f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:02.223331 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296980c6-9c89-4972-a385-ca49cf606828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:02.224069 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296980c6-9c89-4972-a385-ca49cf606828 admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:02.224422 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296980c6-9c89-4972-a385-ca49cf606828 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:02.224768 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296980c6-9c89-4972-a385-ca49cf606828 admin admin] PATH_INFO: `/v3/projects/eb297cdd28bb40ecb134a655266a36cf/users/f39f2e3c85fc44a5b20e4573256c7da9/roles/b75cb360925e4ba5a65f14965c103ffd` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:02.243451 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296980c6-9c89-4972-a385-ca49cf606828 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb297cdd28bb40ecb134a655266a36cf, user_id=f39f2e3c85fc44a5b20e4573256c7da9, role_id=b75cb360925e4ba5a65f14965c103ffd)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:02.245495 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-2b2b6fac-2b0e-475f-8f43-8c46357c662f None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:32:02.246952 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296980c6-9c89-4972-a385-ca49cf606828 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:02.247514 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-2b2b6fac-2b0e-475f-8f43-8c46357c662f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:02.263770 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1034/2085] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:32:01 2022] PUT /identity/v3/projects/eb297cdd28bb40ecb134a655266a36cf/users/f39f2e3c85fc44a5b20e4573256c7da9/roles/b75cb360925e4ba5a65f14965c103ffd => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:32:02.265444 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca16aa6b-685b-42fa-a2a1-bcaaa821a951 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:02.266083 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca16aa6b-685b-42fa-a2a1-bcaaa821a951 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:02.266626 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca16aa6b-685b-42fa-a2a1-bcaaa821a951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:02.269858 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1052/2086] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:32:02 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:32:02.274311 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f45f2a3-85fe-4e0a-8c19-8854962ea2fd None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:02.274701 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f45f2a3-85fe-4e0a-8c19-8854962ea2fd None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:02.275079 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f45f2a3-85fe-4e0a-8c19-8854962ea2fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:02.291030 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-ca16aa6b-685b-42fa-a2a1-bcaaa821a951 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:32:02.292258 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-ca16aa6b-685b-42fa-a2a1-bcaaa821a951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:02.311662 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-8f45f2a3-85fe-4e0a-8c19-8854962ea2fd None None] MFA Rules not processed for user `f39f2e3c85fc44a5b20e4573256c7da9`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:32:02.313421 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-8f45f2a3-85fe-4e0a-8c19-8854962ea2fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:02.354070 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1035/2087] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:32:02 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:32:02.355316 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca16aa6b-685b-42fa-a2a1-bcaaa821a951 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:02.357060 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-ca16aa6b-685b-42fa-a2a1-bcaaa821a951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:02.379484 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1053/2088] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:32:02 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:32:02.380850 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f45f2a3-85fe-4e0a-8c19-8854962ea2fd None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:02.382162 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-8f45f2a3-85fe-4e0a-8c19-8854962ea2fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:02.390101 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:32:02.395793 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f45f2a3-85fe-4e0a-8c19-8854962ea2fd None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:02.400374 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bafca6-e6b3-4fac-8ceb-d16180b22de6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:02.400986 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bafca6-e6b3-4fac-8ceb-d16180b22de6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:02.401264 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bafca6-e6b3-4fac-8ceb-d16180b22de6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:02.401564 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bafca6-e6b3-4fac-8ceb-d16180b22de6 admin admin] PATH_INFO: `/v3/projects/66970187da3d4d56938d16397d7708c5` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:02.402696 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bafca6-e6b3-4fac-8ceb-d16180b22de6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=66970187da3d4d56938d16397d7708c5)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:02.404585 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bafca6-e6b3-4fac-8ceb-d16180b22de6 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:02.416621 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca16aa6b-685b-42fa-a2a1-bcaaa821a951 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:02.420340 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e9537c-899a-4251-965b-73dabf503c65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:02.420934 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e9537c-899a-4251-965b-73dabf503c65 service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:02.421217 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e9537c-899a-4251-965b-73dabf503c65 service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:02.421507 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e9537c-899a-4251-965b-73dabf503c65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:02.430517 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e9537c-899a-4251-965b-73dabf503c65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:02.432068 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e9537c-899a-4251-965b-73dabf503c65 service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:02.442427 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1036/2089] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:32:02 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:32:02.443220 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e9537c-899a-4251-965b-73dabf503c65 service neutron] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:02.451526 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:32:02.453156 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e9537c-899a-4251-965b-73dabf503c65 service neutron] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:02.455702 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-27bafca6-e6b3-4fac-8ceb-d16180b22de6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66970187da3d4d56938d16397d7708c5', 'request_id': 'req-27bafca6-e6b3-4fac-8ceb-d16180b22de6'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:02.470856 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-27bafca6-e6b3-4fac-8ceb-d16180b22de6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66970187da3d4d56938d16397d7708c5', 'request_id': 'req-27bafca6-e6b3-4fac-8ceb-d16180b22de6'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:02.480555 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-27bafca6-e6b3-4fac-8ceb-d16180b22de6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66970187da3d4d56938d16397d7708c5', 'request_id': 'req-27bafca6-e6b3-4fac-8ceb-d16180b22de6'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:02.496482 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-27bafca6-e6b3-4fac-8ceb-d16180b22de6 admin admin] The token cache is being invalidate because project 66970187da3d4d56938d16397d7708c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:32:02.497329 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-27bafca6-e6b3-4fac-8ceb-d16180b22de6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:02.498113 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-27bafca6-e6b3-4fac-8ceb-d16180b22de6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:02.499909 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1054/2090] 10.222.0.102 () {58 vars in 1238 bytes} [Thu Sep 1 17:32:02 2022] DELETE /identity/v3/projects/66970187da3d4d56938d16397d7708c5 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:32:02.502852 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bafca6-e6b3-4fac-8ceb-d16180b22de6 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:02.504997 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-27bafca6-e6b3-4fac-8ceb-d16180b22de6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:02.526148 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b9f026-8d1b-42d4-abbf-59d0fba4ae15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:02.527912 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b9f026-8d1b-42d4-abbf-59d0fba4ae15 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:02.527912 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b9f026-8d1b-42d4-abbf-59d0fba4ae15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:02.527912 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b9f026-8d1b-42d4-abbf-59d0fba4ae15 admin admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:02.528349 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b9f026-8d1b-42d4-abbf-59d0fba4ae15 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:32:02.528677 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b9f026-8d1b-42d4-abbf-59d0fba4ae15 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:02.530289 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b9f026-8d1b-42d4-abbf-59d0fba4ae15 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:02.546337 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1037/2091] 10.222.0.102 () {58 vars in 1158 bytes} [Thu Sep 1 17:32:02 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:32:02.546337 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b9f026-8d1b-42d4-abbf-59d0fba4ae15 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:02.549821 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-19b9f026-8d1b-42d4-abbf-59d0fba4ae15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:02.563611 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b9f026-8d1b-42d4-abbf-59d0fba4ae15 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:02.568407 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc0a76b-4c5e-4121-aebd-a954db995ddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:02.568926 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc0a76b-4c5e-4121-aebd-a954db995ddf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:02.569203 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc0a76b-4c5e-4121-aebd-a954db995ddf admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:02.569503 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc0a76b-4c5e-4121-aebd-a954db995ddf admin admin] PATH_INFO: `/v3/users/5e33ed91311241a3aec54a46880abab3` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:02.570624 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc0a76b-4c5e-4121-aebd-a954db995ddf admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e33ed91311241a3aec54a46880abab3)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:02.572640 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc0a76b-4c5e-4121-aebd-a954db995ddf admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:02.594530 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bafca6-e6b3-4fac-8ceb-d16180b22de6 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:02.610791 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf22a52-f8c2-4365-a2f5-b7aba7b213ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:02.613387 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf22a52-f8c2-4365-a2f5-b7aba7b213ce admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:02.613695 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf22a52-f8c2-4365-a2f5-b7aba7b213ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:02.613967 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf22a52-f8c2-4365-a2f5-b7aba7b213ce admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:02.614598 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf22a52-f8c2-4365-a2f5-b7aba7b213ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:02.616517 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf22a52-f8c2-4365-a2f5-b7aba7b213ce admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:02.633448 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-9fc0a76b-4c5e-4121-aebd-a954db995ddf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e33ed91311241a3aec54a46880abab3'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:02.634191 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-9fc0a76b-4c5e-4121-aebd-a954db995ddf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e33ed91311241a3aec54a46880abab3'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:02.650499 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-9fc0a76b-4c5e-4121-aebd-a954db995ddf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e33ed91311241a3aec54a46880abab3'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:02.657856 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1055/2092] 10.222.0.102 () {60 vars in 1158 bytes} [Thu Sep 1 17:32:02 2022] POST /identity/v3/projects => generated 384 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:32:02.658944 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676517ad-4eea-467e-8d69-3fcbb0555f3e None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:02.659229 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676517ad-4eea-467e-8d69-3fcbb0555f3e None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:02.659491 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676517ad-4eea-467e-8d69-3fcbb0555f3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:02.664296 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-9fc0a76b-4c5e-4121-aebd-a954db995ddf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e33ed91311241a3aec54a46880abab3'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:02.665159 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-9fc0a76b-4c5e-4121-aebd-a954db995ddf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e33ed91311241a3aec54a46880abab3'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:02.683320 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1038/2093] 10.222.0.102 () {58 vars in 1229 bytes} [Thu Sep 1 17:32:02 2022] DELETE /identity/v3/users/5e33ed91311241a3aec54a46880abab3 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:32:02.685870 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc0a76b-4c5e-4121-aebd-a954db995ddf admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:02.687724 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-9fc0a76b-4c5e-4121-aebd-a954db995ddf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:02.688436 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-676517ad-4eea-467e-8d69-3fcbb0555f3e None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:32:02.689825 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-676517ad-4eea-467e-8d69-3fcbb0555f3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:02.738522 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:32:02.769060 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1056/2094] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:32:02 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:32:02.769060 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676517ad-4eea-467e-8d69-3fcbb0555f3e None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:02.769060 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-676517ad-4eea-467e-8d69-3fcbb0555f3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:02.769415 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc0a76b-4c5e-4121-aebd-a954db995ddf admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:02.775664 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f07aa9-7cdd-4451-a575-9932ed15173b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:02.776250 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f07aa9-7cdd-4451-a575-9932ed15173b admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:02.776531 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f07aa9-7cdd-4451-a575-9932ed15173b admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:02.776799 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f07aa9-7cdd-4451-a575-9932ed15173b admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:02.778851 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f07aa9-7cdd-4451-a575-9932ed15173b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:02.780867 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f07aa9-7cdd-4451-a575-9932ed15173b admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:02.792906 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676517ad-4eea-467e-8d69-3fcbb0555f3e None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:02.799608 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f9ef0a-2686-4dfd-9112-92c183fb9a2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:02.800303 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f9ef0a-2686-4dfd-9112-92c183fb9a2e admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:02.800582 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f9ef0a-2686-4dfd-9112-92c183fb9a2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:02.801115 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f9ef0a-2686-4dfd-9112-92c183fb9a2e admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:02.802611 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f9ef0a-2686-4dfd-9112-92c183fb9a2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:02.805508 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f9ef0a-2686-4dfd-9112-92c183fb9a2e admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:02.817465 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1039/2095] 10.222.0.102 () {60 vars in 1158 bytes} [Thu Sep 1 17:32:02 2022] POST /identity/v3/projects => generated 380 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:32:02.818915 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b786cf-35c4-4e60-b3af-67e59213119f None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:02.819234 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b786cf-35c4-4e60-b3af-67e59213119f None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:02.819512 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b786cf-35c4-4e60-b3af-67e59213119f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:02.835882 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1057/2096] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:32:02 2022] POST /identity/v3/users => generated 299 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:32:02.836943 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f9ef0a-2686-4dfd-9112-92c183fb9a2e admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:02.839235 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-07f9ef0a-2686-4dfd-9112-92c183fb9a2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:02.842191 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-e0b786cf-35c4-4e60-b3af-67e59213119f None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:32:02.843465 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-e0b786cf-35c4-4e60-b3af-67e59213119f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:02.853499 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f9ef0a-2686-4dfd-9112-92c183fb9a2e admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:02.896823 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:32:02.910106 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1040/2097] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:32:02 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:32:02.914341 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f9a9cf-06e9-4f1b-8cb7-16069ab18ede None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:02.914626 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f9a9cf-06e9-4f1b-8cb7-16069ab18ede None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:02.914902 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f9a9cf-06e9-4f1b-8cb7-16069ab18ede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:02.925403 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99194c8-71af-41df-97a6-6b8ecdb39eaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:02.926073 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99194c8-71af-41df-97a6-6b8ecdb39eaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:02.926321 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99194c8-71af-41df-97a6-6b8ecdb39eaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:02.926591 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99194c8-71af-41df-97a6-6b8ecdb39eaa admin admin] PATH_INFO: `/v3/projects/5a52942c5d0c4a28a1d13aec1b33840b` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:02.927866 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99194c8-71af-41df-97a6-6b8ecdb39eaa admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a52942c5d0c4a28a1d13aec1b33840b)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:02.930291 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99194c8-71af-41df-97a6-6b8ecdb39eaa admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:02.935991 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-b1f9a9cf-06e9-4f1b-8cb7-16069ab18ede None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:32:02.937186 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-b1f9a9cf-06e9-4f1b-8cb7-16069ab18ede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:02.951963 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1041/2098] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:32:02 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:32:02.951963 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f9a9cf-06e9-4f1b-8cb7-16069ab18ede None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:02.959957 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f9a9cf-06e9-4f1b-8cb7-16069ab18ede None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:02.967550 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a167091f-cda1-4736-8a72-89d9c6dd9d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:02.968164 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a167091f-cda1-4736-8a72-89d9c6dd9d72 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:02.968467 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a167091f-cda1-4736-8a72-89d9c6dd9d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:02.968717 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a167091f-cda1-4736-8a72-89d9c6dd9d72 admin admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:02.969872 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a167091f-cda1-4736-8a72-89d9c6dd9d72 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:02.972207 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a167091f-cda1-4736-8a72-89d9c6dd9d72 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:03.003414 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1042/2099] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:32:02 2022] POST /identity/v3/users => generated 291 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:32:03.003414 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a167091f-cda1-4736-8a72-89d9c6dd9d72 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:03.009175 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-a99194c8-71af-41df-97a6-6b8ecdb39eaa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a52942c5d0c4a28a1d13aec1b33840b', 'request_id': 'req-a99194c8-71af-41df-97a6-6b8ecdb39eaa'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:03.011216 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a167091f-cda1-4736-8a72-89d9c6dd9d72 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:03.019075 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-a99194c8-71af-41df-97a6-6b8ecdb39eaa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a52942c5d0c4a28a1d13aec1b33840b', 'request_id': 'req-a99194c8-71af-41df-97a6-6b8ecdb39eaa'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:03.026645 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-a99194c8-71af-41df-97a6-6b8ecdb39eaa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a52942c5d0c4a28a1d13aec1b33840b', 'request_id': 'req-a99194c8-71af-41df-97a6-6b8ecdb39eaa'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:03.038463 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-a99194c8-71af-41df-97a6-6b8ecdb39eaa admin admin] The token cache is being invalidate because project 5a52942c5d0c4a28a1d13aec1b33840b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:32:03.038970 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-a99194c8-71af-41df-97a6-6b8ecdb39eaa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:03.039816 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-a99194c8-71af-41df-97a6-6b8ecdb39eaa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:03.044409 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1058/2100] 10.222.0.102 () {58 vars in 1238 bytes} [Thu Sep 1 17:32:02 2022] DELETE /identity/v3/projects/5a52942c5d0c4a28a1d13aec1b33840b => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:32:03.044409 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99194c8-71af-41df-97a6-6b8ecdb39eaa admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:03.044409 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-a99194c8-71af-41df-97a6-6b8ecdb39eaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:03.084611 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1a93c2-040b-4159-b348-5ccce5397cf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:03.085223 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1a93c2-040b-4159-b348-5ccce5397cf7 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:03.085528 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1a93c2-040b-4159-b348-5ccce5397cf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:03.085803 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1a93c2-040b-4159-b348-5ccce5397cf7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:03.086511 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1a93c2-040b-4159-b348-5ccce5397cf7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:03.088117 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1a93c2-040b-4159-b348-5ccce5397cf7 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:03.100363 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1043/2101] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:32:03 2022] GET /identity/v3/roles => generated 1427 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:32:03.102961 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1a93c2-040b-4159-b348-5ccce5397cf7 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:03.104750 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-af1a93c2-040b-4159-b348-5ccce5397cf7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:03.113329 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99194c8-71af-41df-97a6-6b8ecdb39eaa admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:03.117682 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e819934e-ee26-4600-9100-eec723a7c0f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:03.118479 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e819934e-ee26-4600-9100-eec723a7c0f7 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:03.118631 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1a93c2-040b-4159-b348-5ccce5397cf7 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:03.118904 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e819934e-ee26-4600-9100-eec723a7c0f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:03.119245 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e819934e-ee26-4600-9100-eec723a7c0f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:03.121082 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-e819934e-ee26-4600-9100-eec723a7c0f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:03.124013 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6425d908-2f20-4b06-bbf9-a8076a1fe8b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:03.124729 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6425d908-2f20-4b06-bbf9-a8076a1fe8b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:03.125068 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6425d908-2f20-4b06-bbf9-a8076a1fe8b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:03.125373 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6425d908-2f20-4b06-bbf9-a8076a1fe8b0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:03.126012 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6425d908-2f20-4b06-bbf9-a8076a1fe8b0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:32:03.126325 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6425d908-2f20-4b06-bbf9-a8076a1fe8b0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:03.127874 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6425d908-2f20-4b06-bbf9-a8076a1fe8b0 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:03.135739 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e819934e-ee26-4600-9100-eec723a7c0f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:03.137996 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e819934e-ee26-4600-9100-eec723a7c0f7 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:03.148257 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1059/2102] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:32:03 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:32:03.149071 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e819934e-ee26-4600-9100-eec723a7c0f7 service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:03.150407 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-e819934e-ee26-4600-9100-eec723a7c0f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:03.154475 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1044/2103] 10.222.0.102 () {58 vars in 1158 bytes} [Thu Sep 1 17:32:03 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:32:03.155544 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6425d908-2f20-4b06-bbf9-a8076a1fe8b0 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:03.159181 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-6425d908-2f20-4b06-bbf9-a8076a1fe8b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:03.162965 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e819934e-ee26-4600-9100-eec723a7c0f7 service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:03.169164 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e2ca62-817c-4f59-8905-1ceb5b9917d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:03.169911 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e2ca62-817c-4f59-8905-1ceb5b9917d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:03.170301 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e2ca62-817c-4f59-8905-1ceb5b9917d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:03.170691 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e2ca62-817c-4f59-8905-1ceb5b9917d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:03.171448 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e2ca62-817c-4f59-8905-1ceb5b9917d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:03.173396 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e2ca62-817c-4f59-8905-1ceb5b9917d5 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:03.186750 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6425d908-2f20-4b06-bbf9-a8076a1fe8b0 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:03.187168 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1060/2104] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:32:03 2022] GET /identity/v3/roles => generated 1427 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:32:03.188364 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e2ca62-817c-4f59-8905-1ceb5b9917d5 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:03.190796 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-55e2ca62-817c-4f59-8905-1ceb5b9917d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:03.191870 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5726c0d6-369f-412c-834a-5e89c0309cca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:03.192629 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5726c0d6-369f-412c-834a-5e89c0309cca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:03.192881 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5726c0d6-369f-412c-834a-5e89c0309cca admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:03.193232 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5726c0d6-369f-412c-834a-5e89c0309cca admin admin] PATH_INFO: `/v3/users/4ddc2f14be0a4b7e96102a7ab76e6cbe` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:03.194509 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5726c0d6-369f-412c-834a-5e89c0309cca admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ddc2f14be0a4b7e96102a7ab76e6cbe)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:03.196713 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5726c0d6-369f-412c-834a-5e89c0309cca admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:03.207145 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e2ca62-817c-4f59-8905-1ceb5b9917d5 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:03.218264 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57871983-91ae-4fe9-94ad-a7a5c914dc52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:03.219342 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:32:03.220471 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57871983-91ae-4fe9-94ad-a7a5c914dc52 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:03.221692 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57871983-91ae-4fe9-94ad-a7a5c914dc52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:03.222065 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57871983-91ae-4fe9-94ad-a7a5c914dc52 admin admin] PATH_INFO: `/v3/projects/ed68084e3b854558930c4b6f11ea07c7/users/f008a95a41164ef682b28212d8f64363/roles/36577a6f46d642d4b81805ede9872490` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:03.244265 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57871983-91ae-4fe9-94ad-a7a5c914dc52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed68084e3b854558930c4b6f11ea07c7, user_id=f008a95a41164ef682b28212d8f64363, role_id=36577a6f46d642d4b81805ede9872490)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:03.247817 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57871983-91ae-4fe9-94ad-a7a5c914dc52 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:03.266338 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1061/2105] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:32:03 2022] PUT /identity/v3/projects/ed68084e3b854558930c4b6f11ea07c7/users/f008a95a41164ef682b28212d8f64363/roles/36577a6f46d642d4b81805ede9872490 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:32:03.267452 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57871983-91ae-4fe9-94ad-a7a5c914dc52 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:03.277973 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-5726c0d6-369f-412c-834a-5e89c0309cca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ddc2f14be0a4b7e96102a7ab76e6cbe'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:03.278927 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-5726c0d6-369f-412c-834a-5e89c0309cca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ddc2f14be0a4b7e96102a7ab76e6cbe'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:03.297189 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-5726c0d6-369f-412c-834a-5e89c0309cca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ddc2f14be0a4b7e96102a7ab76e6cbe'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:03.311225 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-5726c0d6-369f-412c-834a-5e89c0309cca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ddc2f14be0a4b7e96102a7ab76e6cbe'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:03.312357 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-5726c0d6-369f-412c-834a-5e89c0309cca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ddc2f14be0a4b7e96102a7ab76e6cbe'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:03.326666 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1045/2106] 10.222.0.102 () {58 vars in 1229 bytes} [Thu Sep 1 17:32:03 2022] DELETE /identity/v3/users/4ddc2f14be0a4b7e96102a7ab76e6cbe => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:32:03.327817 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5726c0d6-369f-412c-834a-5e89c0309cca admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:03.329292 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-5726c0d6-369f-412c-834a-5e89c0309cca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:03.330530 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:32:03.382076 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57871983-91ae-4fe9-94ad-a7a5c914dc52 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:03.448889 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5726c0d6-369f-412c-834a-5e89c0309cca admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:03.456726 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f96884b-7132-4f79-a01e-a132c9166d47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:03.457653 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f96884b-7132-4f79-a01e-a132c9166d47 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:03.458021 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f96884b-7132-4f79-a01e-a132c9166d47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:03.458357 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f96884b-7132-4f79-a01e-a132c9166d47 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:03.459629 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f96884b-7132-4f79-a01e-a132c9166d47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:03.461864 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f96884b-7132-4f79-a01e-a132c9166d47 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:03.477899 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1046/2107] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:32:03 2022] GET /identity/v3/roles => generated 1427 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:32:03.478752 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f96884b-7132-4f79-a01e-a132c9166d47 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:03.481692 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd752fd-da53-42a0-ab26-5d5952f36045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:03.482463 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd752fd-da53-42a0-ab26-5d5952f36045 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:03.482725 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd752fd-da53-42a0-ab26-5d5952f36045 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:03.483063 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd752fd-da53-42a0-ab26-5d5952f36045 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:03.483755 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd752fd-da53-42a0-ab26-5d5952f36045 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:03.485836 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd752fd-da53-42a0-ab26-5d5952f36045 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:03.487468 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-1f96884b-7132-4f79-a01e-a132c9166d47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:03.506908 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f96884b-7132-4f79-a01e-a132c9166d47 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:03.516684 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f897cd-c4b9-4076-ae57-f7607a637d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:03.517324 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f897cd-c4b9-4076-ae57-f7607a637d2f admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:03.517606 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f897cd-c4b9-4076-ae57-f7607a637d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:03.517924 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f897cd-c4b9-4076-ae57-f7607a637d2f admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:03.518576 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f897cd-c4b9-4076-ae57-f7607a637d2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:03.520189 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f897cd-c4b9-4076-ae57-f7607a637d2f admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:03.531515 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1047/2108] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:32:03 2022] GET /identity/v3/roles => generated 1427 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:32:03.533192 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f897cd-c4b9-4076-ae57-f7607a637d2f admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:03.537422 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1062/2109] 10.222.0.102 () {60 vars in 1158 bytes} [Thu Sep 1 17:32:03 2022] POST /identity/v3/projects => generated 384 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:32:03.537422 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd752fd-da53-42a0-ab26-5d5952f36045 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:03.539856 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-5dd752fd-da53-42a0-ab26-5d5952f36045 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:03.544012 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f897cd-c4b9-4076-ae57-f7607a637d2f admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:03.616437 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93751b7c-c026-4aee-a1dd-83bf889a07ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:03.617211 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93751b7c-c026-4aee-a1dd-83bf889a07ec admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:03.617522 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93751b7c-c026-4aee-a1dd-83bf889a07ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:03.617795 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93751b7c-c026-4aee-a1dd-83bf889a07ec admin admin] PATH_INFO: `/v3/projects/e44d4093cf36429db5f1effcef3d8fbb/users/3c87fd9bfcf7410cb8770c61e520c680/roles/b75cb360925e4ba5a65f14965c103ffd` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:03.625861 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd752fd-da53-42a0-ab26-5d5952f36045 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:03.630730 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49327985-1081-413f-b780-a660c7f71c11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:03.631395 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49327985-1081-413f-b780-a660c7f71c11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:03.631680 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49327985-1081-413f-b780-a660c7f71c11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:03.631954 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49327985-1081-413f-b780-a660c7f71c11 admin admin] PATH_INFO: `/v3/projects/46f445ab9e364c44a1e45c25f6ec679a` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:03.633390 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49327985-1081-413f-b780-a660c7f71c11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=46f445ab9e364c44a1e45c25f6ec679a)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:03.635313 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49327985-1081-413f-b780-a660c7f71c11 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:03.636797 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93751b7c-c026-4aee-a1dd-83bf889a07ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=e44d4093cf36429db5f1effcef3d8fbb, user_id=3c87fd9bfcf7410cb8770c61e520c680, role_id=b75cb360925e4ba5a65f14965c103ffd)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:03.639549 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93751b7c-c026-4aee-a1dd-83bf889a07ec admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:03.666750 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1048/2110] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:32:03 2022] PUT /identity/v3/projects/e44d4093cf36429db5f1effcef3d8fbb/users/3c87fd9bfcf7410cb8770c61e520c680/roles/b75cb360925e4ba5a65f14965c103ffd => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:32:03.666750 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93751b7c-c026-4aee-a1dd-83bf889a07ec admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:03.666750 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:32:03.704457 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-49327985-1081-413f-b780-a660c7f71c11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46f445ab9e364c44a1e45c25f6ec679a', 'request_id': 'req-49327985-1081-413f-b780-a660c7f71c11'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:03.718147 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-49327985-1081-413f-b780-a660c7f71c11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46f445ab9e364c44a1e45c25f6ec679a', 'request_id': 'req-49327985-1081-413f-b780-a660c7f71c11'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:03.727242 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-49327985-1081-413f-b780-a660c7f71c11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46f445ab9e364c44a1e45c25f6ec679a', 'request_id': 'req-49327985-1081-413f-b780-a660c7f71c11'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:03.735673 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:32:03.740311 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-49327985-1081-413f-b780-a660c7f71c11 admin admin] The token cache is being invalidate because project 46f445ab9e364c44a1e45c25f6ec679a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:32:03.740614 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-49327985-1081-413f-b780-a660c7f71c11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:03.741714 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-49327985-1081-413f-b780-a660c7f71c11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:03.743935 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1063/2111] 10.222.0.102 () {58 vars in 1238 bytes} [Thu Sep 1 17:32:03 2022] DELETE /identity/v3/projects/46f445ab9e364c44a1e45c25f6ec679a => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:32:03.744812 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49327985-1081-413f-b780-a660c7f71c11 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:03.746125 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-49327985-1081-413f-b780-a660c7f71c11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:03.774558 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93751b7c-c026-4aee-a1dd-83bf889a07ec admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:03.908491 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49327985-1081-413f-b780-a660c7f71c11 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:03.928462 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac15381-3595-4e6c-8dc8-d54843544809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:03.931942 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8fa1e8-b959-49d8-982a-b3ba9e785e32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:03.933042 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac15381-3595-4e6c-8dc8-d54843544809 admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:03.933619 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8fa1e8-b959-49d8-982a-b3ba9e785e32 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:03.936228 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8fa1e8-b959-49d8-982a-b3ba9e785e32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:03.936397 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac15381-3595-4e6c-8dc8-d54843544809 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:03.937173 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac15381-3595-4e6c-8dc8-d54843544809 admin admin] PATH_INFO: `/v3/projects/ed68084e3b854558930c4b6f11ea07c7/users/f008a95a41164ef682b28212d8f64363/roles/36577a6f46d642d4b81805ede9872490` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:03.937326 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8fa1e8-b959-49d8-982a-b3ba9e785e32 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:03.938119 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8fa1e8-b959-49d8-982a-b3ba9e785e32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:03.942042 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac15381-3595-4e6c-8dc8-d54843544809 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed68084e3b854558930c4b6f11ea07c7, user_id=f008a95a41164ef682b28212d8f64363, role_id=36577a6f46d642d4b81805ede9872490)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:03.946279 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8fa1e8-b959-49d8-982a-b3ba9e785e32 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:03.950489 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac15381-3595-4e6c-8dc8-d54843544809 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:03.986113 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1064/2112] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:32:03 2022] POST /identity/v3/users => generated 293 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:32:03.986113 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c5a4d0-0875-4b32-9936-e013a07a6f28 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:03.986113 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c5a4d0-0875-4b32-9936-e013a07a6f28 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:03.986113 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c5a4d0-0875-4b32-9936-e013a07a6f28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:03.986569 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1049/2113] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:32:03 2022] PUT /identity/v3/projects/ed68084e3b854558930c4b6f11ea07c7/users/f008a95a41164ef682b28212d8f64363/roles/36577a6f46d642d4b81805ede9872490 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:32:03.986569 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac15381-3595-4e6c-8dc8-d54843544809 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:03.990370 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-bac15381-3595-4e6c-8dc8-d54843544809 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:04.045026 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-67c5a4d0-0875-4b32-9936-e013a07a6f28 None None] MFA Rules not processed for user `3c87fd9bfcf7410cb8770c61e520c680`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:32:04.050430 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-67c5a4d0-0875-4b32-9936-e013a07a6f28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:04.146877 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac15381-3595-4e6c-8dc8-d54843544809 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:04.155874 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae90c8be-a27b-4111-a3cb-21abf57c1633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:04.157103 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae90c8be-a27b-4111-a3cb-21abf57c1633 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:04.157724 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae90c8be-a27b-4111-a3cb-21abf57c1633 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:04.158248 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae90c8be-a27b-4111-a3cb-21abf57c1633 admin admin] PATH_INFO: `/v3/projects/0c93afe3b7c84ce28a4b0f5f9f3dbf8f` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:04.159517 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae90c8be-a27b-4111-a3cb-21abf57c1633 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c93afe3b7c84ce28a4b0f5f9f3dbf8f)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:04.161756 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae90c8be-a27b-4111-a3cb-21abf57c1633 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:04.212086 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1065/2114] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:32:03 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:32:04.212956 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c5a4d0-0875-4b32-9936-e013a07a6f28 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:04.214755 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-67c5a4d0-0875-4b32-9936-e013a07a6f28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:04.231570 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c5a4d0-0875-4b32-9936-e013a07a6f28 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:04.233674 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:32:04.238486 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-ae90c8be-a27b-4111-a3cb-21abf57c1633 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c93afe3b7c84ce28a4b0f5f9f3dbf8f', 'request_id': 'req-ae90c8be-a27b-4111-a3cb-21abf57c1633'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:04.256561 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-ae90c8be-a27b-4111-a3cb-21abf57c1633 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c93afe3b7c84ce28a4b0f5f9f3dbf8f', 'request_id': 'req-ae90c8be-a27b-4111-a3cb-21abf57c1633'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:04.271549 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-ae90c8be-a27b-4111-a3cb-21abf57c1633 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c93afe3b7c84ce28a4b0f5f9f3dbf8f', 'request_id': 'req-ae90c8be-a27b-4111-a3cb-21abf57c1633'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:04.295072 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-ae90c8be-a27b-4111-a3cb-21abf57c1633 admin admin] The token cache is being invalidate because project 0c93afe3b7c84ce28a4b0f5f9f3dbf8f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:32:04.295578 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-ae90c8be-a27b-4111-a3cb-21abf57c1633 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:04.296838 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-ae90c8be-a27b-4111-a3cb-21abf57c1633 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:04.299263 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1050/2115] 10.222.0.102 () {58 vars in 1238 bytes} [Thu Sep 1 17:32:03 2022] DELETE /identity/v3/projects/0c93afe3b7c84ce28a4b0f5f9f3dbf8f => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:32:04.300310 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae90c8be-a27b-4111-a3cb-21abf57c1633 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:04.301953 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-ae90c8be-a27b-4111-a3cb-21abf57c1633 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:04.348232 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4cc1a0-62aa-4e34-a8c2-056516b49f23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:04.349009 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4cc1a0-62aa-4e34-a8c2-056516b49f23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:04.349465 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4cc1a0-62aa-4e34-a8c2-056516b49f23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:04.349787 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4cc1a0-62aa-4e34-a8c2-056516b49f23 admin admin] PATH_INFO: `/v3/users/ac66fa5f10024cb8b0f0a093856f9dc9` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:04.351228 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4cc1a0-62aa-4e34-a8c2-056516b49f23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac66fa5f10024cb8b0f0a093856f9dc9)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:04.354258 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4cc1a0-62aa-4e34-a8c2-056516b49f23 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:04.386138 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:32:04.414188 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae90c8be-a27b-4111-a3cb-21abf57c1633 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:04.427622 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-3a4cc1a0-62aa-4e34-a8c2-056516b49f23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac66fa5f10024cb8b0f0a093856f9dc9'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:04.428434 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e673f27-be28-4e7c-9072-c35c5a1dbc43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:04.429419 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e673f27-be28-4e7c-9072-c35c5a1dbc43 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:04.429887 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e673f27-be28-4e7c-9072-c35c5a1dbc43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:04.430686 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e673f27-be28-4e7c-9072-c35c5a1dbc43 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:04.431502 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e673f27-be28-4e7c-9072-c35c5a1dbc43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:04.434544 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e673f27-be28-4e7c-9072-c35c5a1dbc43 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:04.451857 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1051/2116] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:32:04 2022] GET /identity/v3/roles => generated 1427 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:32:04.455958 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-3a4cc1a0-62aa-4e34-a8c2-056516b49f23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac66fa5f10024cb8b0f0a093856f9dc9'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:04.456750 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6506c094-e3f6-4ff7-b98a-f60e76442ea4 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:04.457331 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6506c094-e3f6-4ff7-b98a-f60e76442ea4 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:04.458189 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6506c094-e3f6-4ff7-b98a-f60e76442ea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:04.476108 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-3a4cc1a0-62aa-4e34-a8c2-056516b49f23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac66fa5f10024cb8b0f0a093856f9dc9'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:04.477779 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-3a4cc1a0-62aa-4e34-a8c2-056516b49f23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac66fa5f10024cb8b0f0a093856f9dc9'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:04.478412 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-3a4cc1a0-62aa-4e34-a8c2-056516b49f23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac66fa5f10024cb8b0f0a093856f9dc9'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:04.509578 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1066/2117] 10.222.0.102 () {58 vars in 1229 bytes} [Thu Sep 1 17:32:04 2022] DELETE /identity/v3/users/ac66fa5f10024cb8b0f0a093856f9dc9 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:32:04.510527 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4cc1a0-62aa-4e34-a8c2-056516b49f23 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:04.514036 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-6506c094-e3f6-4ff7-b98a-f60e76442ea4 None None] MFA Rules not processed for user `f008a95a41164ef682b28212d8f64363`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:32:04.514508 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-3a4cc1a0-62aa-4e34-a8c2-056516b49f23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:04.515498 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-6506c094-e3f6-4ff7-b98a-f60e76442ea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:04.617576 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4cc1a0-62aa-4e34-a8c2-056516b49f23 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:04.624578 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407ab1cc-fcb0-4769-8113-752c6a9e2fa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:04.625332 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407ab1cc-fcb0-4769-8113-752c6a9e2fa9 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:04.625692 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407ab1cc-fcb0-4769-8113-752c6a9e2fa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:04.626066 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407ab1cc-fcb0-4769-8113-752c6a9e2fa9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:04.626889 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407ab1cc-fcb0-4769-8113-752c6a9e2fa9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:04.629127 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407ab1cc-fcb0-4769-8113-752c6a9e2fa9 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:04.652035 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1052/2118] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:32:04 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:32:04.654822 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86d994c-129b-4e65-af25-8010e0c1e159 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:04.655287 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86d994c-129b-4e65-af25-8010e0c1e159 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:04.656189 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86d994c-129b-4e65-af25-8010e0c1e159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:04.678403 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1067/2119] 10.222.0.102 () {60 vars in 1158 bytes} [Thu Sep 1 17:32:04 2022] POST /identity/v3/projects => generated 382 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:32:04.681711 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407ab1cc-fcb0-4769-8113-752c6a9e2fa9 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:04.685610 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-407ab1cc-fcb0-4769-8113-752c6a9e2fa9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:04.692814 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-b86d994c-129b-4e65-af25-8010e0c1e159 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:32:04.694346 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-b86d994c-129b-4e65-af25-8010e0c1e159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:04.764469 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:32:04.787505 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1053/2120] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:32:04 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:32:04.788397 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86d994c-129b-4e65-af25-8010e0c1e159 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:04.792270 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-b86d994c-129b-4e65-af25-8010e0c1e159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:04.816002 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407ab1cc-fcb0-4769-8113-752c6a9e2fa9 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:04.823983 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec7815e-39c1-4a9c-bb10-c190460e0998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:04.826511 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec7815e-39c1-4a9c-bb10-c190460e0998 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:04.827071 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec7815e-39c1-4a9c-bb10-c190460e0998 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:04.827601 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec7815e-39c1-4a9c-bb10-c190460e0998 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:04.828911 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec7815e-39c1-4a9c-bb10-c190460e0998 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:04.831137 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec7815e-39c1-4a9c-bb10-c190460e0998 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:04.845622 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1068/2121] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:32:04 2022] GET /identity/v3/roles => generated 1427 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:32:04.858526 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec7815e-39c1-4a9c-bb10-c190460e0998 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:04.877922 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec7815e-39c1-4a9c-bb10-c190460e0998 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:04.879735 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86d994c-129b-4e65-af25-8010e0c1e159 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:04.884661 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d101eb-5400-49b6-8a1b-0c862b6dbdcc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '523ed589872a45a09b6a7675a7fb0b75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:04.885277 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d101eb-5400-49b6-8a1b-0c862b6dbdcc service cinder] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:04.885931 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d101eb-5400-49b6-8a1b-0c862b6dbdcc service cinder] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:04.886211 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d101eb-5400-49b6-8a1b-0c862b6dbdcc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:04.888066 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-a4d101eb-5400-49b6-8a1b-0c862b6dbdcc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:04.892739 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6e2155-85e2-4931-ad1a-1c51e97a7aa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:04.893556 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6e2155-85e2-4931-ad1a-1c51e97a7aa0 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:04.893933 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6e2155-85e2-4931-ad1a-1c51e97a7aa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:04.896749 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6e2155-85e2-4931-ad1a-1c51e97a7aa0 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:04.898412 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6e2155-85e2-4931-ad1a-1c51e97a7aa0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:04.904507 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6e2155-85e2-4931-ad1a-1c51e97a7aa0 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:04.954953 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1069/2122] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:32:04 2022] POST /identity/v3/users => generated 298 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:32:04.956741 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e879a9f0-cbf6-4246-9bf7-ee0bd68afeb7 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:04.957212 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e879a9f0-cbf6-4246-9bf7-ee0bd68afeb7 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:04.958121 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e879a9f0-cbf6-4246-9bf7-ee0bd68afeb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:04.968430 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:DELETE=7,keystone:SELECT=88,keystone:INSERT=5 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:32:04.986151 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d101eb-5400-49b6-8a1b-0c862b6dbdcc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:04.988092 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d101eb-5400-49b6-8a1b-0c862b6dbdcc service cinder] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:04.991485 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-e879a9f0-cbf6-4246-9bf7-ee0bd68afeb7 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:32:04.993195 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-e879a9f0-cbf6-4246-9bf7-ee0bd68afeb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:05.002166 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1054/2123] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:32:04 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:32:05.003102 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d101eb-5400-49b6-8a1b-0c862b6dbdcc service cinder] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:05.003665 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1070/2124] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:32:04 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:32:05.004544 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e879a9f0-cbf6-4246-9bf7-ee0bd68afeb7 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:05.015666 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e879a9f0-cbf6-4246-9bf7-ee0bd68afeb7 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:05.015950 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d101eb-5400-49b6-8a1b-0c862b6dbdcc service cinder] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:05.021625 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51fcec1-3cf8-40ec-b036-7f37f578b958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:05.022356 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51fcec1-3cf8-40ec-b036-7f37f578b958 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:05.022649 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51fcec1-3cf8-40ec-b036-7f37f578b958 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:05.022928 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51fcec1-3cf8-40ec-b036-7f37f578b958 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:05.023587 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51fcec1-3cf8-40ec-b036-7f37f578b958 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:05.025200 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51fcec1-3cf8-40ec-b036-7f37f578b958 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:05.035732 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df39f65-c1e1-4481-8be6-a81813343ea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:05.036510 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df39f65-c1e1-4481-8be6-a81813343ea0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:05.037347 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df39f65-c1e1-4481-8be6-a81813343ea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:05.037783 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1071/2125] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:32:05 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:32:05.038091 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df39f65-c1e1-4481-8be6-a81813343ea0 admin admin] PATH_INFO: `/v3/projects/5f340b6490a9427abee7ef0c60de08f2/users/0f83be6352184bcab5f842eb7739ec3b/roles/b75cb360925e4ba5a65f14965c103ffd` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:05.038772 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51fcec1-3cf8-40ec-b036-7f37f578b958 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:05.052857 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51fcec1-3cf8-40ec-b036-7f37f578b958 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:05.059875 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df39f65-c1e1-4481-8be6-a81813343ea0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f340b6490a9427abee7ef0c60de08f2, user_id=0f83be6352184bcab5f842eb7739ec3b, role_id=b75cb360925e4ba5a65f14965c103ffd)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:05.060696 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315edd0f-fb00-49d0-84a0-437f4f3e3435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:05.061669 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315edd0f-fb00-49d0-84a0-437f4f3e3435 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:05.062190 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315edd0f-fb00-49d0-84a0-437f4f3e3435 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:05.062624 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315edd0f-fb00-49d0-84a0-437f4f3e3435 admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:05.063267 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df39f65-c1e1-4481-8be6-a81813343ea0 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:05.063405 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315edd0f-fb00-49d0-84a0-437f4f3e3435 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:32:05.063725 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315edd0f-fb00-49d0-84a0-437f4f3e3435 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:05.065419 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315edd0f-fb00-49d0-84a0-437f4f3e3435 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:05.083509 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1072/2126] 10.222.0.102 () {58 vars in 1158 bytes} [Thu Sep 1 17:32:05 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:32:05.083898 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1055/2127] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:32:05 2022] PUT /identity/v3/projects/5f340b6490a9427abee7ef0c60de08f2/users/0f83be6352184bcab5f842eb7739ec3b/roles/b75cb360925e4ba5a65f14965c103ffd => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:32:05.084360 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315edd0f-fb00-49d0-84a0-437f4f3e3435 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:05.087925 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df39f65-c1e1-4481-8be6-a81813343ea0 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:05.097408 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315edd0f-fb00-49d0-84a0-437f4f3e3435 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:05.168228 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dd0406-5c75-4abc-ae94-e89bc44e43a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:05.169234 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dd0406-5c75-4abc-ae94-e89bc44e43a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:05.169603 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dd0406-5c75-4abc-ae94-e89bc44e43a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:05.169938 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dd0406-5c75-4abc-ae94-e89bc44e43a9 admin admin] PATH_INFO: `/v3/projects/e2b5d89bac10408a86f4ccedc7fac802/users/11b32e1193054ce6b83b016562757c4f/roles/36577a6f46d642d4b81805ede9872490` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:05.184770 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df39f65-c1e1-4481-8be6-a81813343ea0 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:05.189695 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d808b6f1-6dfb-4088-ae0f-5d3e9880923f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:05.190305 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d808b6f1-6dfb-4088-ae0f-5d3e9880923f admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:05.190660 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d808b6f1-6dfb-4088-ae0f-5d3e9880923f admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:05.190849 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d808b6f1-6dfb-4088-ae0f-5d3e9880923f admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:05.191634 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d808b6f1-6dfb-4088-ae0f-5d3e9880923f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:05.194159 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d808b6f1-6dfb-4088-ae0f-5d3e9880923f admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:05.205199 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dd0406-5c75-4abc-ae94-e89bc44e43a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2b5d89bac10408a86f4ccedc7fac802, user_id=11b32e1193054ce6b83b016562757c4f, role_id=36577a6f46d642d4b81805ede9872490)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:05.209995 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dd0406-5c75-4abc-ae94-e89bc44e43a9 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:05.228191 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1073/2128] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:32:05 2022] PUT /identity/v3/projects/e2b5d89bac10408a86f4ccedc7fac802/users/11b32e1193054ce6b83b016562757c4f/roles/36577a6f46d642d4b81805ede9872490 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:32:05.233871 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900b5aaf-3d45-4e94-bec4-e1a6a2cb8aa8 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:05.233871 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900b5aaf-3d45-4e94-bec4-e1a6a2cb8aa8 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:05.234295 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900b5aaf-3d45-4e94-bec4-e1a6a2cb8aa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:05.247625 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1056/2129] 10.222.0.102 () {60 vars in 1158 bytes} [Thu Sep 1 17:32:05 2022] POST /identity/v3/projects => generated 378 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:32:05.247625 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d808b6f1-6dfb-4088-ae0f-5d3e9880923f admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:05.251117 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-d808b6f1-6dfb-4088-ae0f-5d3e9880923f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:05.267956 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-900b5aaf-3d45-4e94-bec4-e1a6a2cb8aa8 None None] MFA Rules not processed for user `0f83be6352184bcab5f842eb7739ec3b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:32:05.269187 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-900b5aaf-3d45-4e94-bec4-e1a6a2cb8aa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:05.278517 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:32:05.347906 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d808b6f1-6dfb-4088-ae0f-5d3e9880923f admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:05.348074 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1074/2130] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:32:05 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:32:05.348074 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900b5aaf-3d45-4e94-bec4-e1a6a2cb8aa8 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:05.350999 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f9a7b7-7e1b-4e85-9242-7c72176f94a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:05.351692 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f9a7b7-7e1b-4e85-9242-7c72176f94a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:05.351983 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f9a7b7-7e1b-4e85-9242-7c72176f94a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:05.352384 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f9a7b7-7e1b-4e85-9242-7c72176f94a6 admin admin] PATH_INFO: `/v3/projects/e810acd39e804c379756ee1c9b2ae995` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:05.366176 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f9a7b7-7e1b-4e85-9242-7c72176f94a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e810acd39e804c379756ee1c9b2ae995)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:05.366176 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f9a7b7-7e1b-4e85-9242-7c72176f94a6 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:05.366548 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900b5aaf-3d45-4e94-bec4-e1a6a2cb8aa8 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:05.366548 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6f8e23-d819-4d9e-b783-85cd5567a50f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:05.368256 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6f8e23-d819-4d9e-b783-85cd5567a50f admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:05.368658 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6f8e23-d819-4d9e-b783-85cd5567a50f admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:05.369694 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6f8e23-d819-4d9e-b783-85cd5567a50f admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:05.370236 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6f8e23-d819-4d9e-b783-85cd5567a50f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:05.372132 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6f8e23-d819-4d9e-b783-85cd5567a50f admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:05.387901 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1075/2131] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:32:05 2022] GET /identity/v3/roles => generated 1427 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:32:05.389030 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6f8e23-d819-4d9e-b783-85cd5567a50f admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:05.401399 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6f8e23-d819-4d9e-b783-85cd5567a50f admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:05.425725 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-55f9a7b7-7e1b-4e85-9242-7c72176f94a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e810acd39e804c379756ee1c9b2ae995', 'request_id': 'req-55f9a7b7-7e1b-4e85-9242-7c72176f94a6'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:05.432765 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-55f9a7b7-7e1b-4e85-9242-7c72176f94a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e810acd39e804c379756ee1c9b2ae995', 'request_id': 'req-55f9a7b7-7e1b-4e85-9242-7c72176f94a6'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:05.442227 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-55f9a7b7-7e1b-4e85-9242-7c72176f94a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e810acd39e804c379756ee1c9b2ae995', 'request_id': 'req-55f9a7b7-7e1b-4e85-9242-7c72176f94a6'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:05.446014 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:32:05.456534 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-55f9a7b7-7e1b-4e85-9242-7c72176f94a6 admin admin] The token cache is being invalidate because project e810acd39e804c379756ee1c9b2ae995 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:32:05.456916 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-55f9a7b7-7e1b-4e85-9242-7c72176f94a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:05.457688 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-55f9a7b7-7e1b-4e85-9242-7c72176f94a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:05.459791 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1057/2132] 10.222.0.102 () {58 vars in 1238 bytes} [Thu Sep 1 17:32:05 2022] DELETE /identity/v3/projects/e810acd39e804c379756ee1c9b2ae995 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:32:05.460767 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f9a7b7-7e1b-4e85-9242-7c72176f94a6 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:05.462747 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-55f9a7b7-7e1b-4e85-9242-7c72176f94a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:05.480989 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445a58f7-6c5a-4499-bb0a-b6f2d33ee3b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:05.481832 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445a58f7-6c5a-4499-bb0a-b6f2d33ee3b5 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:05.482131 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445a58f7-6c5a-4499-bb0a-b6f2d33ee3b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:05.482427 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445a58f7-6c5a-4499-bb0a-b6f2d33ee3b5 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:05.483099 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445a58f7-6c5a-4499-bb0a-b6f2d33ee3b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:05.485380 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445a58f7-6c5a-4499-bb0a-b6f2d33ee3b5 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:05.511748 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1076/2133] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:32:05 2022] POST /identity/v3/users => generated 290 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:32:05.513291 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445a58f7-6c5a-4499-bb0a-b6f2d33ee3b5 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:05.515184 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-445a58f7-6c5a-4499-bb0a-b6f2d33ee3b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:05.528701 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445a58f7-6c5a-4499-bb0a-b6f2d33ee3b5 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:05.533640 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b3536e-588c-4e01-819f-a40a9b5d9c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:05.534297 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b3536e-588c-4e01-819f-a40a9b5d9c4b admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:05.534578 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b3536e-588c-4e01-819f-a40a9b5d9c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:05.534853 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b3536e-588c-4e01-819f-a40a9b5d9c4b admin admin] PATH_INFO: `/v3/projects/e2b5d89bac10408a86f4ccedc7fac802/users/11b32e1193054ce6b83b016562757c4f/roles/36577a6f46d642d4b81805ede9872490` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:05.536843 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b3536e-588c-4e01-819f-a40a9b5d9c4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2b5d89bac10408a86f4ccedc7fac802, user_id=11b32e1193054ce6b83b016562757c4f, role_id=36577a6f46d642d4b81805ede9872490)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:05.539543 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b3536e-588c-4e01-819f-a40a9b5d9c4b admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:05.544099 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f9a7b7-7e1b-4e85-9242-7c72176f94a6 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:05.549716 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bf6515-3d6b-41af-ac54-6caad6c04036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:05.550347 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bf6515-3d6b-41af-ac54-6caad6c04036 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:05.550627 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bf6515-3d6b-41af-ac54-6caad6c04036 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:05.550898 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bf6515-3d6b-41af-ac54-6caad6c04036 admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:05.551549 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bf6515-3d6b-41af-ac54-6caad6c04036 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:05.552293 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1077/2134] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:32:05 2022] PUT /identity/v3/projects/e2b5d89bac10408a86f4ccedc7fac802/users/11b32e1193054ce6b83b016562757c4f/roles/36577a6f46d642d4b81805ede9872490 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:32:05.553699 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fb30f5-2eb4-44d0-9a1f-74dc18aa562a None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:05.553985 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fb30f5-2eb4-44d0-9a1f-74dc18aa562a None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:05.554374 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fb30f5-2eb4-44d0-9a1f-74dc18aa562a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:05.558584 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bf6515-3d6b-41af-ac54-6caad6c04036 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:05.577885 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-a4fb30f5-2eb4-44d0-9a1f-74dc18aa562a None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:32:05.579320 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-a4fb30f5-2eb4-44d0-9a1f-74dc18aa562a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:05.592419 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1058/2135] 10.222.0.102 () {60 vars in 1158 bytes} [Thu Sep 1 17:32:05 2022] POST /identity/v3/projects => generated 382 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:32:05.593434 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bf6515-3d6b-41af-ac54-6caad6c04036 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:05.595197 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-36bf6515-3d6b-41af-ac54-6caad6c04036 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:05.627213 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:32:05.652250 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1078/2136] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:32:05 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:32:05.654271 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7862a4-0101-4ee4-b329-07a48a52def3 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:05.654768 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7862a4-0101-4ee4-b329-07a48a52def3 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:05.654955 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7862a4-0101-4ee4-b329-07a48a52def3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:05.676825 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bf6515-3d6b-41af-ac54-6caad6c04036 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:05.681695 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e956ebfa-59ff-42cc-94d5-e4b0a130a4f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:05.682391 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e956ebfa-59ff-42cc-94d5-e4b0a130a4f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:05.682741 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e956ebfa-59ff-42cc-94d5-e4b0a130a4f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:05.683085 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e956ebfa-59ff-42cc-94d5-e4b0a130a4f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:05.683849 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e956ebfa-59ff-42cc-94d5-e4b0a130a4f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:05.684717 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-7e7862a4-0101-4ee4-b329-07a48a52def3 None None] MFA Rules not processed for user `11b32e1193054ce6b83b016562757c4f`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:32:05.685611 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e956ebfa-59ff-42cc-94d5-e4b0a130a4f2 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:05.685968 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-7e7862a4-0101-4ee4-b329-07a48a52def3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:05.695913 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1059/2137] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:32:05 2022] GET /identity/v3/roles => generated 1427 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:32:05.696662 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e956ebfa-59ff-42cc-94d5-e4b0a130a4f2 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:05.706527 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e956ebfa-59ff-42cc-94d5-e4b0a130a4f2 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:05.711192 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e33b4c-3acd-4cd1-8201-09358d64d88e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:05.711870 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e33b4c-3acd-4cd1-8201-09358d64d88e admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:05.712158 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e33b4c-3acd-4cd1-8201-09358d64d88e admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:05.712438 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e33b4c-3acd-4cd1-8201-09358d64d88e admin admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:05.713055 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e33b4c-3acd-4cd1-8201-09358d64d88e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:05.715002 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e33b4c-3acd-4cd1-8201-09358d64d88e admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:05.741403 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1060/2138] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:32:05 2022] POST /identity/v3/users => generated 298 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:32:05.741403 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f324dc8-126d-42db-a417-f94e8cfc5652 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:05.741403 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f324dc8-126d-42db-a417-f94e8cfc5652 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:05.741403 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f324dc8-126d-42db-a417-f94e8cfc5652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:05.765409 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-6f324dc8-126d-42db-a417-f94e8cfc5652 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:32:05.766639 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-6f324dc8-126d-42db-a417-f94e8cfc5652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:05.767870 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1079/2139] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:32:05 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:32:05.768852 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7862a4-0101-4ee4-b329-07a48a52def3 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:05.778784 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1061/2140] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:32:05 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:32:05.778784 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f324dc8-126d-42db-a417-f94e8cfc5652 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:05.784857 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7862a4-0101-4ee4-b329-07a48a52def3 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:05.788034 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f324dc8-126d-42db-a417-f94e8cfc5652 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:05.790448 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0ee2d3-45dd-4a07-be53-68ed64f23c8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:05.791277 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0ee2d3-45dd-4a07-be53-68ed64f23c8f admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:05.791712 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0ee2d3-45dd-4a07-be53-68ed64f23c8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:05.792100 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0ee2d3-45dd-4a07-be53-68ed64f23c8f admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:05.792851 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0ee2d3-45dd-4a07-be53-68ed64f23c8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:05.793960 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae857a6-6c8d-4865-a5aa-0dd7abe49857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:05.794597 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae857a6-6c8d-4865-a5aa-0dd7abe49857 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:05.794882 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae857a6-6c8d-4865-a5aa-0dd7abe49857 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:05.795155 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae857a6-6c8d-4865-a5aa-0dd7abe49857 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:05.795727 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0ee2d3-45dd-4a07-be53-68ed64f23c8f admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:05.795842 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae857a6-6c8d-4865-a5aa-0dd7abe49857 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:05.797381 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae857a6-6c8d-4865-a5aa-0dd7abe49857 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:05.810912 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1062/2142] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:32:05 2022] GET /identity/v3/roles => generated 1427 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:32:05.811121 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1080/2142] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:32:05 2022] GET /identity/v3/roles => generated 1427 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:32:05.811217 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae857a6-6c8d-4865-a5aa-0dd7abe49857 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:05.811819 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0ee2d3-45dd-4a07-be53-68ed64f23c8f admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:05.812241 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-3ae857a6-6c8d-4865-a5aa-0dd7abe49857 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:05.828214 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0ee2d3-45dd-4a07-be53-68ed64f23c8f admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:05.835208 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364ff130-2a3d-4df2-94d7-45f178eee87d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:05.835835 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364ff130-2a3d-4df2-94d7-45f178eee87d admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:05.836101 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364ff130-2a3d-4df2-94d7-45f178eee87d admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:05.836369 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364ff130-2a3d-4df2-94d7-45f178eee87d admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:05.837015 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364ff130-2a3d-4df2-94d7-45f178eee87d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:32:05.837348 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364ff130-2a3d-4df2-94d7-45f178eee87d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:05.838937 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364ff130-2a3d-4df2-94d7-45f178eee87d admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:05.863136 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1081/2143] 10.222.0.102 () {58 vars in 1158 bytes} [Thu Sep 1 17:32:05 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:32:05.863914 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364ff130-2a3d-4df2-94d7-45f178eee87d admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:05.880111 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364ff130-2a3d-4df2-94d7-45f178eee87d admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:05.886870 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae41f51-f256-42ad-a10c-7649e0d3842d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:05.888100 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae41f51-f256-42ad-a10c-7649e0d3842d admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:05.888776 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae41f51-f256-42ad-a10c-7649e0d3842d admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:05.888776 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae41f51-f256-42ad-a10c-7649e0d3842d admin admin] PATH_INFO: `/v3/projects/f7b968ff5d004f42a053a8971a6ee361/users/ffa6c849eb85444093b76fcf388d0676/roles/36577a6f46d642d4b81805ede9872490` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:05.901362 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:32:05.910263 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae41f51-f256-42ad-a10c-7649e0d3842d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7b968ff5d004f42a053a8971a6ee361, user_id=ffa6c849eb85444093b76fcf388d0676, role_id=36577a6f46d642d4b81805ede9872490)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:05.913011 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae41f51-f256-42ad-a10c-7649e0d3842d admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:05.926486 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae857a6-6c8d-4865-a5aa-0dd7abe49857 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:05.930641 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1082/2144] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:32:05 2022] PUT /identity/v3/projects/f7b968ff5d004f42a053a8971a6ee361/users/ffa6c849eb85444093b76fcf388d0676/roles/36577a6f46d642d4b81805ede9872490 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:32:05.932139 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae41f51-f256-42ad-a10c-7649e0d3842d admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:05.990522 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b47844-2c37-4052-aa36-d5d4dd06f8bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '523ed589872a45a09b6a7675a7fb0b75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:05.991383 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b47844-2c37-4052-aa36-d5d4dd06f8bb service cinder] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:05.991747 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b47844-2c37-4052-aa36-d5d4dd06f8bb service cinder] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:05.991923 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b47844-2c37-4052-aa36-d5d4dd06f8bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:06.006996 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae41f51-f256-42ad-a10c-7649e0d3842d admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:06.012520 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ee69bf-65d0-4d7b-ada8-5a29f4fd536a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:06.013182 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ee69bf-65d0-4d7b-ada8-5a29f4fd536a admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:06.013677 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ee69bf-65d0-4d7b-ada8-5a29f4fd536a admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:06.013954 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ee69bf-65d0-4d7b-ada8-5a29f4fd536a admin admin] PATH_INFO: `/v3/projects/74914f5328ea4a9ebde30423869af79f/users/25e2c68661dc421ca27eccdbea6a7f93/roles/b75cb360925e4ba5a65f14965c103ffd` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:06.030366 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ee69bf-65d0-4d7b-ada8-5a29f4fd536a admin admin] RBAC: Authorizing `identity:create_grant(project_id=74914f5328ea4a9ebde30423869af79f, user_id=25e2c68661dc421ca27eccdbea6a7f93, role_id=b75cb360925e4ba5a65f14965c103ffd)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:06.033241 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ee69bf-65d0-4d7b-ada8-5a29f4fd536a admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:06.049146 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1083/2145] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:32:05 2022] PUT /identity/v3/projects/74914f5328ea4a9ebde30423869af79f/users/25e2c68661dc421ca27eccdbea6a7f93/roles/b75cb360925e4ba5a65f14965c103ffd => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:32:06.050152 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ee69bf-65d0-4d7b-ada8-5a29f4fd536a admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:06.068330 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:32:06.088874 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b47844-2c37-4052-aa36-d5d4dd06f8bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:06.093679 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b47844-2c37-4052-aa36-d5d4dd06f8bb service cinder] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:06.118987 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1063/2146] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:32:05 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:32:06.119950 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b47844-2c37-4052-aa36-d5d4dd06f8bb service cinder] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:06.133015 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ee69bf-65d0-4d7b-ada8-5a29f4fd536a admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:06.139485 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef203dc-e143-4c3e-825f-40252fc4dc95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:06.140224 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef203dc-e143-4c3e-825f-40252fc4dc95 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:06.140510 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef203dc-e143-4c3e-825f-40252fc4dc95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:06.140785 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef203dc-e143-4c3e-825f-40252fc4dc95 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:06.140948 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b47844-2c37-4052-aa36-d5d4dd06f8bb service cinder] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:06.141460 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef203dc-e143-4c3e-825f-40252fc4dc95 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:06.144180 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef203dc-e143-4c3e-825f-40252fc4dc95 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:06.146469 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9922c426-e811-4f8d-a8f8-e442db6f28f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:06.147025 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9922c426-e811-4f8d-a8f8-e442db6f28f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:06.147478 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9922c426-e811-4f8d-a8f8-e442db6f28f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:06.147659 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9922c426-e811-4f8d-a8f8-e442db6f28f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:06.148495 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9922c426-e811-4f8d-a8f8-e442db6f28f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:06.150348 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9922c426-e811-4f8d-a8f8-e442db6f28f3 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:06.164668 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1064/2147] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:32:06 2022] GET /identity/v3/roles => generated 1427 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:32:06.164668 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e001da7-d492-4f1f-8393-0b298feac708 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:06.164668 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e001da7-d492-4f1f-8393-0b298feac708 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:06.164668 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e001da7-d492-4f1f-8393-0b298feac708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:06.187084 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1084/2148] 10.222.0.102 () {60 vars in 1158 bytes} [Thu Sep 1 17:32:06 2022] POST /identity/v3/projects => generated 376 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:32:06.189230 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef203dc-e143-4c3e-825f-40252fc4dc95 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:06.224286 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-3e001da7-d492-4f1f-8393-0b298feac708 None None] MFA Rules not processed for user `25e2c68661dc421ca27eccdbea6a7f93`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:32:06.225939 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-3e001da7-d492-4f1f-8393-0b298feac708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:06.285869 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:32:06.295681 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef203dc-e143-4c3e-825f-40252fc4dc95 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:06.307910 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd0fd8e-1223-4e5d-9d72-aec6997287e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:06.309633 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd0fd8e-1223-4e5d-9d72-aec6997287e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:06.310401 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd0fd8e-1223-4e5d-9d72-aec6997287e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:06.312268 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd0fd8e-1223-4e5d-9d72-aec6997287e8 admin admin] PATH_INFO: `/v3/projects/f7b968ff5d004f42a053a8971a6ee361/users/ffa6c849eb85444093b76fcf388d0676/roles/36577a6f46d642d4b81805ede9872490` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:06.314971 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd0fd8e-1223-4e5d-9d72-aec6997287e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7b968ff5d004f42a053a8971a6ee361, user_id=ffa6c849eb85444093b76fcf388d0676, role_id=36577a6f46d642d4b81805ede9872490)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:06.319554 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd0fd8e-1223-4e5d-9d72-aec6997287e8 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:06.325443 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1065/2149] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:32:06 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:32:06.325443 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e001da7-d492-4f1f-8393-0b298feac708 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:06.337964 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e001da7-d492-4f1f-8393-0b298feac708 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:06.344226 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3aba12-9d5e-479e-9308-8a0b5373d6c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:06.344870 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3aba12-9d5e-479e-9308-8a0b5373d6c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:06.345112 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3aba12-9d5e-479e-9308-8a0b5373d6c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:06.345523 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3aba12-9d5e-479e-9308-8a0b5373d6c2 admin admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:06.346582 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3aba12-9d5e-479e-9308-8a0b5373d6c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:06.347808 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1085/2150] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:32:06 2022] PUT /identity/v3/projects/f7b968ff5d004f42a053a8971a6ee361/users/ffa6c849eb85444093b76fcf388d0676/roles/36577a6f46d642d4b81805ede9872490 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:32:06.350314 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd0fd8e-1223-4e5d-9d72-aec6997287e8 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:06.360818 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3aba12-9d5e-479e-9308-8a0b5373d6c2 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:06.368176 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-4fd0fd8e-1223-4e5d-9d72-aec6997287e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:06.410689 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1066/2151] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:32:06 2022] POST /identity/v3/users => generated 289 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:32:06.410689 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd1c7e3-ffd3-42a3-82ec-69988a7d9c7c None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:06.410689 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd1c7e3-ffd3-42a3-82ec-69988a7d9c7c None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:06.410689 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd1c7e3-ffd3-42a3-82ec-69988a7d9c7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:06.466183 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-5dd1c7e3-ffd3-42a3-82ec-69988a7d9c7c None None] MFA Rules not processed for user `ffa6c849eb85444093b76fcf388d0676`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:32:06.468167 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-5dd1c7e3-ffd3-42a3-82ec-69988a7d9c7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:06.589422 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd0fd8e-1223-4e5d-9d72-aec6997287e8 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:06.595426 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b6873d-c825-46b9-9905-3f87501c34ae service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '039683b8d1d146c6add5ffa235d06f91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:06.596385 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b6873d-c825-46b9-9905-3f87501c34ae service glance] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:06.596680 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b6873d-c825-46b9-9905-3f87501c34ae service glance] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:06.597372 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b6873d-c825-46b9-9905-3f87501c34ae service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:06.615179 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1067/2152] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:32:06 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:32:06.617008 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd1c7e3-ffd3-42a3-82ec-69988a7d9c7c None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:06.674563 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b6873d-c825-46b9-9905-3f87501c34ae service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:06.676644 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b6873d-c825-46b9-9905-3f87501c34ae service glance] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:06.678151 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:32:06.696460 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1086/2153] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:32:06 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:32:06.697854 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b6873d-c825-46b9-9905-3f87501c34ae service glance] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:06.699649 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-a8b6873d-c825-46b9-9905-3f87501c34ae service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:06.706188 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd1c7e3-ffd3-42a3-82ec-69988a7d9c7c None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:06.711732 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617b54d1-a169-4dd0-8aef-c31f201d0809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:06.712417 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617b54d1-a169-4dd0-8aef-c31f201d0809 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:06.727921 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617b54d1-a169-4dd0-8aef-c31f201d0809 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:06.727921 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617b54d1-a169-4dd0-8aef-c31f201d0809 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:06.727921 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617b54d1-a169-4dd0-8aef-c31f201d0809 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:06.727921 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617b54d1-a169-4dd0-8aef-c31f201d0809 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:06.731750 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1068/2154] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:32:06 2022] GET /identity/v3/roles => generated 1427 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:32:06.732765 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617b54d1-a169-4dd0-8aef-c31f201d0809 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:06.747714 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617b54d1-a169-4dd0-8aef-c31f201d0809 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:06.753857 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba38f1c8-81ac-4278-9f24-5aeaaff3916e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:06.754810 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba38f1c8-81ac-4278-9f24-5aeaaff3916e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:06.755101 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba38f1c8-81ac-4278-9f24-5aeaaff3916e admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:06.755471 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba38f1c8-81ac-4278-9f24-5aeaaff3916e admin admin] PATH_INFO: `/v3/users/3c87fd9bfcf7410cb8770c61e520c680` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:06.756644 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba38f1c8-81ac-4278-9f24-5aeaaff3916e admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c87fd9bfcf7410cb8770c61e520c680)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:06.758828 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba38f1c8-81ac-4278-9f24-5aeaaff3916e admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:06.772330 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b6873d-c825-46b9-9905-3f87501c34ae service glance] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:06.780990 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39d362b-8a11-4792-b281-0dc121721273 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '523ed589872a45a09b6a7675a7fb0b75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:06.780990 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39d362b-8a11-4792-b281-0dc121721273 service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:06.780990 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39d362b-8a11-4792-b281-0dc121721273 service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:06.780990 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39d362b-8a11-4792-b281-0dc121721273 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:06.792193 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39d362b-8a11-4792-b281-0dc121721273 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:06.794710 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39d362b-8a11-4792-b281-0dc121721273 service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:06.816971 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1087/2155] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:32:06 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:32:06.822385 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39d362b-8a11-4792-b281-0dc121721273 service cinder] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:06.842062 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39d362b-8a11-4792-b281-0dc121721273 service cinder] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:06.845710 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-ba38f1c8-81ac-4278-9f24-5aeaaff3916e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c87fd9bfcf7410cb8770c61e520c680'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:06.846953 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-ba38f1c8-81ac-4278-9f24-5aeaaff3916e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c87fd9bfcf7410cb8770c61e520c680'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:06.861252 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6706e2-e599-4f5b-a070-7208a937f42a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:06.861897 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6706e2-e599-4f5b-a070-7208a937f42a admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:06.862179 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6706e2-e599-4f5b-a070-7208a937f42a admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:06.862501 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6706e2-e599-4f5b-a070-7208a937f42a admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:06.863781 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6706e2-e599-4f5b-a070-7208a937f42a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:06.873011 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6706e2-e599-4f5b-a070-7208a937f42a admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:06.894848 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1088/2156] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:32:06 2022] GET /identity/v3/roles => generated 1427 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:32:06.895003 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-ba38f1c8-81ac-4278-9f24-5aeaaff3916e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c87fd9bfcf7410cb8770c61e520c680'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:06.896713 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6706e2-e599-4f5b-a070-7208a937f42a admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:06.905235 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-ba38f1c8-81ac-4278-9f24-5aeaaff3916e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c87fd9bfcf7410cb8770c61e520c680'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:06.906352 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-ba38f1c8-81ac-4278-9f24-5aeaaff3916e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c87fd9bfcf7410cb8770c61e520c680'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:06.910272 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6706e2-e599-4f5b-a070-7208a937f42a admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:06.916301 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551bd998-0f4b-47e0-9165-9c71e67d7f12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:06.917338 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551bd998-0f4b-47e0-9165-9c71e67d7f12 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:06.917767 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551bd998-0f4b-47e0-9165-9c71e67d7f12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:06.918736 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551bd998-0f4b-47e0-9165-9c71e67d7f12 admin admin] PATH_INFO: `/v3/projects/2dd0527a96e948b58c3a728fd0afb070/users/a3dd1402040e4796ac35015d03f44481/roles/b75cb360925e4ba5a65f14965c103ffd` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:06.935830 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1069/2157] 10.222.0.102 () {58 vars in 1229 bytes} [Thu Sep 1 17:32:06 2022] DELETE /identity/v3/users/3c87fd9bfcf7410cb8770c61e520c680 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:32:06.938118 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba38f1c8-81ac-4278-9f24-5aeaaff3916e admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:06.941265 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:32:06.941692 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-ba38f1c8-81ac-4278-9f24-5aeaaff3916e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:06.945558 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551bd998-0f4b-47e0-9165-9c71e67d7f12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2dd0527a96e948b58c3a728fd0afb070, user_id=a3dd1402040e4796ac35015d03f44481, role_id=b75cb360925e4ba5a65f14965c103ffd)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:06.951471 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551bd998-0f4b-47e0-9165-9c71e67d7f12 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:06.978531 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1089/2158] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:32:06 2022] PUT /identity/v3/projects/2dd0527a96e948b58c3a728fd0afb070/users/a3dd1402040e4796ac35015d03f44481/roles/b75cb360925e4ba5a65f14965c103ffd => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:32:06.978531 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c4579f-2b66-456a-aaca-d82e53eb0d08 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:06.978531 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c4579f-2b66-456a-aaca-d82e53eb0d08 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:06.978531 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c4579f-2b66-456a-aaca-d82e53eb0d08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:06.999233 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-32c4579f-2b66-456a-aaca-d82e53eb0d08 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:32:07.001516 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-32c4579f-2b66-456a-aaca-d82e53eb0d08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:07.024702 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba38f1c8-81ac-4278-9f24-5aeaaff3916e admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:07.080599 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1090/2159] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:32:06 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:32:07.082325 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbf4a3d-4798-4fdf-ac60-7f9dcc319413 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:07.082804 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbf4a3d-4798-4fdf-ac60-7f9dcc319413 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:07.083305 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbf4a3d-4798-4fdf-ac60-7f9dcc319413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:07.112141 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bed522a-fa2e-44e0-95fb-46c3df93133c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '523ed589872a45a09b6a7675a7fb0b75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:07.112752 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bed522a-fa2e-44e0-95fb-46c3df93133c service cinder] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:07.113037 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bed522a-fa2e-44e0-95fb-46c3df93133c service cinder] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:07.113332 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bed522a-fa2e-44e0-95fb-46c3df93133c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:07.115145 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-6bed522a-fa2e-44e0-95fb-46c3df93133c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:07.132755 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-7fbf4a3d-4798-4fdf-ac60-7f9dcc319413 None None] MFA Rules not processed for user `a3dd1402040e4796ac35015d03f44481`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:32:07.134822 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-7fbf4a3d-4798-4fdf-ac60-7f9dcc319413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:07.162189 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:32:07.254501 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bed522a-fa2e-44e0-95fb-46c3df93133c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:07.255964 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bed522a-fa2e-44e0-95fb-46c3df93133c service cinder] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:07.293435 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1070/2160] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:32:06 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:32:07.294593 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bed522a-fa2e-44e0-95fb-46c3df93133c service cinder] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:07.298728 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-6bed522a-fa2e-44e0-95fb-46c3df93133c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:07.334829 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1091/2161] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:32:07 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:32:07.422853 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbf4a3d-4798-4fdf-ac60-7f9dcc319413 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:07.424361 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-7fbf4a3d-4798-4fdf-ac60-7f9dcc319413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:07.436242 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bed522a-fa2e-44e0-95fb-46c3df93133c service cinder] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:07.442888 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d412ac-942d-4937-9114-11f67b253721 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:07.443317 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbf4a3d-4798-4fdf-ac60-7f9dcc319413 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:07.445413 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d412ac-942d-4937-9114-11f67b253721 service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:07.446039 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d412ac-942d-4937-9114-11f67b253721 service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:07.446527 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d412ac-942d-4937-9114-11f67b253721 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:07.469357 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d412ac-942d-4937-9114-11f67b253721 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:07.471440 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d412ac-942d-4937-9114-11f67b253721 service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:07.472817 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492bbbf3-a224-4ea9-94f0-849a9eb00b74 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '523ed589872a45a09b6a7675a7fb0b75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:07.473488 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492bbbf3-a224-4ea9-94f0-849a9eb00b74 service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:07.473770 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492bbbf3-a224-4ea9-94f0-849a9eb00b74 service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:07.474043 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492bbbf3-a224-4ea9-94f0-849a9eb00b74 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:07.488595 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1071/2162] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:32:07 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:32:07.498607 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492bbbf3-a224-4ea9-94f0-849a9eb00b74 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:07.500156 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492bbbf3-a224-4ea9-94f0-849a9eb00b74 service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:07.521802 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1092/2163] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:32:07 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 17:32:07.918929 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d412ac-942d-4937-9114-11f67b253721 service neutron] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:07.929583 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d412ac-942d-4937-9114-11f67b253721 service neutron] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:07.933863 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f73496c-e416-4bec-87c9-52eb810a57c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '523ed589872a45a09b6a7675a7fb0b75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:07.934466 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f73496c-e416-4bec-87c9-52eb810a57c5 service cinder] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:07.934986 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f73496c-e416-4bec-87c9-52eb810a57c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:07.935410 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f73496c-e416-4bec-87c9-52eb810a57c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:07.937505 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-0f73496c-e416-4bec-87c9-52eb810a57c5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:08.061688 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f73496c-e416-4bec-87c9-52eb810a57c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:08.063285 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f73496c-e416-4bec-87c9-52eb810a57c5 service cinder] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:08.077876 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1072/2164] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:32:07 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:32:08.238886 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492bbbf3-a224-4ea9-94f0-849a9eb00b74 service cinder] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:08.240309 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-492bbbf3-a224-4ea9-94f0-849a9eb00b74 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:08.258021 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492bbbf3-a224-4ea9-94f0-849a9eb00b74 service cinder] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:08.264920 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53375c3c-e7fa-4a1a-8d49-820ce9862fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:08.266356 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53375c3c-e7fa-4a1a-8d49-820ce9862fb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:08.266581 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53375c3c-e7fa-4a1a-8d49-820ce9862fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:08.266703 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f73496c-e416-4bec-87c9-52eb810a57c5 service cinder] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:08.266895 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53375c3c-e7fa-4a1a-8d49-820ce9862fb0 admin admin] PATH_INFO: `/v3/projects/e44d4093cf36429db5f1effcef3d8fbb` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:08.268465 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53375c3c-e7fa-4a1a-8d49-820ce9862fb0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e44d4093cf36429db5f1effcef3d8fbb)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:08.281527 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53375c3c-e7fa-4a1a-8d49-820ce9862fb0 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:08.285996 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f73496c-e416-4bec-87c9-52eb810a57c5 service cinder] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:08.290006 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f969f5-6a34-470c-bc00-e5440176aa43 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '523ed589872a45a09b6a7675a7fb0b75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:08.290675 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f969f5-6a34-470c-bc00-e5440176aa43 service cinder] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:08.291037 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f969f5-6a34-470c-bc00-e5440176aa43 service cinder] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:08.291223 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f969f5-6a34-470c-bc00-e5440176aa43 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:08.293175 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-39f969f5-6a34-470c-bc00-e5440176aa43 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:08.366017 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-53375c3c-e7fa-4a1a-8d49-820ce9862fb0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e44d4093cf36429db5f1effcef3d8fbb', 'request_id': 'req-53375c3c-e7fa-4a1a-8d49-820ce9862fb0'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:08.375291 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-53375c3c-e7fa-4a1a-8d49-820ce9862fb0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e44d4093cf36429db5f1effcef3d8fbb', 'request_id': 'req-53375c3c-e7fa-4a1a-8d49-820ce9862fb0'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:08.383502 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:32:08.383841 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-53375c3c-e7fa-4a1a-8d49-820ce9862fb0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e44d4093cf36429db5f1effcef3d8fbb', 'request_id': 'req-53375c3c-e7fa-4a1a-8d49-820ce9862fb0'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:08.387559 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:32:08.396155 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f969f5-6a34-470c-bc00-e5440176aa43 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:08.398641 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f969f5-6a34-470c-bc00-e5440176aa43 service cinder] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:08.415907 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-53375c3c-e7fa-4a1a-8d49-820ce9862fb0 admin admin] The token cache is being invalidate because project e44d4093cf36429db5f1effcef3d8fbb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:32:08.416309 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-53375c3c-e7fa-4a1a-8d49-820ce9862fb0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:08.418209 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-53375c3c-e7fa-4a1a-8d49-820ce9862fb0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:08.424201 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1093/2165] 10.222.0.102 () {58 vars in 1238 bytes} [Thu Sep 1 17:32:08 2022] DELETE /identity/v3/projects/e44d4093cf36429db5f1effcef3d8fbb => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:32:08.427840 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53375c3c-e7fa-4a1a-8d49-820ce9862fb0 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:08.432665 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-53375c3c-e7fa-4a1a-8d49-820ce9862fb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:08.437063 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1073/2166] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:32:08 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 17:32:08.578742 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53375c3c-e7fa-4a1a-8d49-820ce9862fb0 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:08.584565 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096fe4ab-4177-451d-bb00-fa79612536cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:08.585326 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096fe4ab-4177-451d-bb00-fa79612536cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:08.585714 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096fe4ab-4177-451d-bb00-fa79612536cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:08.586086 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096fe4ab-4177-451d-bb00-fa79612536cf admin admin] PATH_INFO: `/v3/users/11b32e1193054ce6b83b016562757c4f` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:08.587369 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096fe4ab-4177-451d-bb00-fa79612536cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=11b32e1193054ce6b83b016562757c4f)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:08.589332 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096fe4ab-4177-451d-bb00-fa79612536cf admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:08.636736 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-096fe4ab-4177-451d-bb00-fa79612536cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11b32e1193054ce6b83b016562757c4f'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:08.651997 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-096fe4ab-4177-451d-bb00-fa79612536cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11b32e1193054ce6b83b016562757c4f'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:08.659522 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-096fe4ab-4177-451d-bb00-fa79612536cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11b32e1193054ce6b83b016562757c4f'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:08.660191 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-096fe4ab-4177-451d-bb00-fa79612536cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11b32e1193054ce6b83b016562757c4f'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:08.660825 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-096fe4ab-4177-451d-bb00-fa79612536cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11b32e1193054ce6b83b016562757c4f'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:08.673723 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1094/2167] 10.222.0.102 () {58 vars in 1229 bytes} [Thu Sep 1 17:32:08 2022] DELETE /identity/v3/users/11b32e1193054ce6b83b016562757c4f => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:32:09.023772 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f969f5-6a34-470c-bc00-e5440176aa43 service cinder] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:09.039046 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f969f5-6a34-470c-bc00-e5440176aa43 service cinder] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:09.161452 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c6f163-3ebd-4ea5-83a3-5bec3f800698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:09.161967 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c6f163-3ebd-4ea5-83a3-5bec3f800698 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:09.162257 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c6f163-3ebd-4ea5-83a3-5bec3f800698 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:09.162536 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c6f163-3ebd-4ea5-83a3-5bec3f800698 admin admin] PATH_INFO: `/v3/projects/e2b5d89bac10408a86f4ccedc7fac802` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:09.163685 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c6f163-3ebd-4ea5-83a3-5bec3f800698 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2b5d89bac10408a86f4ccedc7fac802)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:09.165680 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c6f163-3ebd-4ea5-83a3-5bec3f800698 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:09.243979 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-c3c6f163-3ebd-4ea5-83a3-5bec3f800698 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2b5d89bac10408a86f4ccedc7fac802', 'request_id': 'req-c3c6f163-3ebd-4ea5-83a3-5bec3f800698'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:09.250865 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-c3c6f163-3ebd-4ea5-83a3-5bec3f800698 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2b5d89bac10408a86f4ccedc7fac802', 'request_id': 'req-c3c6f163-3ebd-4ea5-83a3-5bec3f800698'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:09.264403 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-c3c6f163-3ebd-4ea5-83a3-5bec3f800698 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2b5d89bac10408a86f4ccedc7fac802', 'request_id': 'req-c3c6f163-3ebd-4ea5-83a3-5bec3f800698'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:09.287570 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-c3c6f163-3ebd-4ea5-83a3-5bec3f800698 admin admin] The token cache is being invalidate because project e2b5d89bac10408a86f4ccedc7fac802 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:32:09.288622 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-c3c6f163-3ebd-4ea5-83a3-5bec3f800698 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:09.295686 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-c3c6f163-3ebd-4ea5-83a3-5bec3f800698 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:09.303710 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3598d8c8-a861-49d5-8ba4-f51551d92b09 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:09.304351 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3598d8c8-a861-49d5-8ba4-f51551d92b09 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:09.304853 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3598d8c8-a861-49d5-8ba4-f51551d92b09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:09.310355 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1074/2168] 10.222.0.102 () {58 vars in 1238 bytes} [Thu Sep 1 17:32:09 2022] DELETE /identity/v3/projects/e2b5d89bac10408a86f4ccedc7fac802 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:32:09.329287 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-3598d8c8-a861-49d5-8ba4-f51551d92b09 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:32:09.330920 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-3598d8c8-a861-49d5-8ba4-f51551d92b09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:09.420638 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1095/2169] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:32:09 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:32:09.431741 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbbce61-67de-4c93-8bef-957a45419288 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:09.432124 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbbce61-67de-4c93-8bef-957a45419288 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:09.432576 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbbce61-67de-4c93-8bef-957a45419288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:09.459285 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-9cbbce61-67de-4c93-8bef-957a45419288 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:32:09.460506 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-9cbbce61-67de-4c93-8bef-957a45419288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:09.488193 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1075/2170] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:32:09 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:32:09.492327 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3598d8c8-a861-49d5-8ba4-f51551d92b09 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:09.504864 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3598d8c8-a861-49d5-8ba4-f51551d92b09 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:09.516181 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114acf1e-c346-40da-849d-1660501253a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:09.517862 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114acf1e-c346-40da-849d-1660501253a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:09.519503 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114acf1e-c346-40da-849d-1660501253a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:09.522278 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114acf1e-c346-40da-849d-1660501253a1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:09.523090 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114acf1e-c346-40da-849d-1660501253a1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:32:09.523504 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114acf1e-c346-40da-849d-1660501253a1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:09.530258 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114acf1e-c346-40da-849d-1660501253a1 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:09.553456 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:32:09.567834 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbbce61-67de-4c93-8bef-957a45419288 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:09.570026 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1096/2171] 10.222.0.102 () {58 vars in 1158 bytes} [Thu Sep 1 17:32:09 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:32:09.597601 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbbce61-67de-4c93-8bef-957a45419288 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:09.605670 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3333f71-23b1-4531-816c-a7fe31aef0b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:09.606293 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3333f71-23b1-4531-816c-a7fe31aef0b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:09.606573 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3333f71-23b1-4531-816c-a7fe31aef0b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:09.606841 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3333f71-23b1-4531-816c-a7fe31aef0b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:09.607707 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3333f71-23b1-4531-816c-a7fe31aef0b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:09.609696 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3333f71-23b1-4531-816c-a7fe31aef0b6 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:09.655026 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1076/2172] 10.222.0.102 () {60 vars in 1158 bytes} [Thu Sep 1 17:32:09 2022] POST /identity/v3/projects => generated 374 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:32:09.656876 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114acf1e-c346-40da-849d-1660501253a1 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:09.669522 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114acf1e-c346-40da-849d-1660501253a1 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:09.739077 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2489ea-9e24-407a-baf9-aae63fbe4f1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:09.739686 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2489ea-9e24-407a-baf9-aae63fbe4f1b admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:09.740063 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2489ea-9e24-407a-baf9-aae63fbe4f1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:09.740345 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2489ea-9e24-407a-baf9-aae63fbe4f1b admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:09.740982 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2489ea-9e24-407a-baf9-aae63fbe4f1b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:09.743030 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2489ea-9e24-407a-baf9-aae63fbe4f1b admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:09.768373 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1097/2173] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:32:09 2022] POST /identity/v3/users => generated 288 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:32:09.770077 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3333f71-23b1-4531-816c-a7fe31aef0b6 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:09.780567 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3333f71-23b1-4531-816c-a7fe31aef0b6 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:09.785480 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0e8924-168f-452e-8a95-c7cdb3d3433c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:09.786170 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0e8924-168f-452e-8a95-c7cdb3d3433c admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:09.786455 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0e8924-168f-452e-8a95-c7cdb3d3433c admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:09.786730 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0e8924-168f-452e-8a95-c7cdb3d3433c admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:09.787423 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0e8924-168f-452e-8a95-c7cdb3d3433c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:09.789006 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0e8924-168f-452e-8a95-c7cdb3d3433c admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:09.801749 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1077/2174] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:32:09 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:32:09.803911 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2489ea-9e24-407a-baf9-aae63fbe4f1b admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:09.818314 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2489ea-9e24-407a-baf9-aae63fbe4f1b admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:09.825687 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d60bb8-1209-41df-9cca-8ecc193ba963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:09.826405 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d60bb8-1209-41df-9cca-8ecc193ba963 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:09.826691 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d60bb8-1209-41df-9cca-8ecc193ba963 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:09.826967 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d60bb8-1209-41df-9cca-8ecc193ba963 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:09.827612 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d60bb8-1209-41df-9cca-8ecc193ba963 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:09.829226 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d60bb8-1209-41df-9cca-8ecc193ba963 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:09.840467 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1098/2175] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:32:09 2022] GET /identity/v3/roles => generated 1427 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:32:09.845037 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0e8924-168f-452e-8a95-c7cdb3d3433c admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:09.856180 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0e8924-168f-452e-8a95-c7cdb3d3433c admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:09.862190 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9304c860-b6d2-455a-a5d4-2197c9af881a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:09.862865 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9304c860-b6d2-455a-a5d4-2197c9af881a admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:09.863145 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9304c860-b6d2-455a-a5d4-2197c9af881a admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:09.863425 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9304c860-b6d2-455a-a5d4-2197c9af881a admin admin] PATH_INFO: `/v3/projects/0ba49950cc214014a48b1829c862a8d4/users/d7bd83f1f7e24df2890ab3fc1aabb60d/roles/b75cb360925e4ba5a65f14965c103ffd` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:09.880042 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9304c860-b6d2-455a-a5d4-2197c9af881a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ba49950cc214014a48b1829c862a8d4, user_id=d7bd83f1f7e24df2890ab3fc1aabb60d, role_id=b75cb360925e4ba5a65f14965c103ffd)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:09.882873 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9304c860-b6d2-455a-a5d4-2197c9af881a admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:09.897238 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1078/2176] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:32:09 2022] PUT /identity/v3/projects/0ba49950cc214014a48b1829c862a8d4/users/d7bd83f1f7e24df2890ab3fc1aabb60d/roles/b75cb360925e4ba5a65f14965c103ffd => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:32:09.902589 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918f2646-d37b-40d5-bc7a-1d6da298c761 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:09.902883 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918f2646-d37b-40d5-bc7a-1d6da298c761 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:09.903162 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918f2646-d37b-40d5-bc7a-1d6da298c761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:09.923235 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-918f2646-d37b-40d5-bc7a-1d6da298c761 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:32:09.924445 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-918f2646-d37b-40d5-bc7a-1d6da298c761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:09.987373 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1099/2177] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:32:09 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:32:09.997750 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9304c860-b6d2-455a-a5d4-2197c9af881a admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:09.999696 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-9304c860-b6d2-455a-a5d4-2197c9af881a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:10.030029 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:32:10.071084 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9304c860-b6d2-455a-a5d4-2197c9af881a admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:10.075163 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a56045-b547-4375-a0ce-c5d2b035516c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:10.075771 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a56045-b547-4375-a0ce-c5d2b035516c service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:10.076046 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a56045-b547-4375-a0ce-c5d2b035516c service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:10.076315 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a56045-b547-4375-a0ce-c5d2b035516c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:10.086230 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a56045-b547-4375-a0ce-c5d2b035516c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:10.087690 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a56045-b547-4375-a0ce-c5d2b035516c service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:10.098493 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1079/2178] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:32:09 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 17:32:10.943577 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b6a8bb-c16f-43e0-bb55-ab50b2bc4117 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:10.944214 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b6a8bb-c16f-43e0-bb55-ab50b2bc4117 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:10.944214 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b6a8bb-c16f-43e0-bb55-ab50b2bc4117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:10.997936 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-b7b6a8bb-c16f-43e0-bb55-ab50b2bc4117 None None] MFA Rules not processed for user `d7bd83f1f7e24df2890ab3fc1aabb60d`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:32:10.999240 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-b7b6a8bb-c16f-43e0-bb55-ab50b2bc4117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:11.190915 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1100/2179] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:32:10 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:32:11.192127 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a56045-b547-4375-a0ce-c5d2b035516c service neutron] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:11.208468 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a56045-b547-4375-a0ce-c5d2b035516c service neutron] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:11.239890 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de6cca2-a9da-4dcd-b6b9-27e11192e999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:11.240634 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de6cca2-a9da-4dcd-b6b9-27e11192e999 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:11.241091 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de6cca2-a9da-4dcd-b6b9-27e11192e999 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:11.243666 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de6cca2-a9da-4dcd-b6b9-27e11192e999 admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:11.244411 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de6cca2-a9da-4dcd-b6b9-27e11192e999 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:11.246741 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de6cca2-a9da-4dcd-b6b9-27e11192e999 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:11.287750 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1080/2180] 10.222.0.102 () {60 vars in 1158 bytes} [Thu Sep 1 17:32:11 2022] POST /identity/v3/projects => generated 372 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:32:11.291743 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b6a8bb-c16f-43e0-bb55-ab50b2bc4117 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:11.302524 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b6a8bb-c16f-43e0-bb55-ab50b2bc4117 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:11.306005 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:32:11.365793 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba63814-25b4-4bcf-9fc6-c6409eece793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:11.366448 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba63814-25b4-4bcf-9fc6-c6409eece793 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:11.366805 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba63814-25b4-4bcf-9fc6-c6409eece793 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:11.367252 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba63814-25b4-4bcf-9fc6-c6409eece793 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:11.368071 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba63814-25b4-4bcf-9fc6-c6409eece793 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:11.370543 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba63814-25b4-4bcf-9fc6-c6409eece793 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:11.397655 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1101/2181] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:32:11 2022] POST /identity/v3/users => generated 293 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:32:11.401689 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de6cca2-a9da-4dcd-b6b9-27e11192e999 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:11.413803 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de6cca2-a9da-4dcd-b6b9-27e11192e999 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:11.418461 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb294dd-9bfb-4be5-924f-6a5596a435bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:11.419065 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb294dd-9bfb-4be5-924f-6a5596a435bf admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:11.419346 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb294dd-9bfb-4be5-924f-6a5596a435bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:11.419619 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb294dd-9bfb-4be5-924f-6a5596a435bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:11.420251 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb294dd-9bfb-4be5-924f-6a5596a435bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:11.421922 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb294dd-9bfb-4be5-924f-6a5596a435bf admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:11.432418 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1081/2182] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:32:11 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 17:32:11.437859 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba63814-25b4-4bcf-9fc6-c6409eece793 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:11.447707 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba63814-25b4-4bcf-9fc6-c6409eece793 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:11.452923 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59dcc2a-80aa-49b1-943b-ee9ee0c5afd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:11.453603 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59dcc2a-80aa-49b1-943b-ee9ee0c5afd9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:11.453883 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59dcc2a-80aa-49b1-943b-ee9ee0c5afd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:11.454183 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59dcc2a-80aa-49b1-943b-ee9ee0c5afd9 admin admin] PATH_INFO: `/v3/projects/57bbcfd7a34d41d3b27d332c5adede46/users/4b04c3d415514ec38d4b28d19d8504e2/roles/36577a6f46d642d4b81805ede9872490` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:11.470141 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59dcc2a-80aa-49b1-943b-ee9ee0c5afd9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57bbcfd7a34d41d3b27d332c5adede46, user_id=4b04c3d415514ec38d4b28d19d8504e2, role_id=36577a6f46d642d4b81805ede9872490)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:11.473135 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59dcc2a-80aa-49b1-943b-ee9ee0c5afd9 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:11.486909 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1102/2183] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:32:11 2022] PUT /identity/v3/projects/57bbcfd7a34d41d3b27d332c5adede46/users/4b04c3d415514ec38d4b28d19d8504e2/roles/36577a6f46d642d4b81805ede9872490 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:32:11.490969 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb294dd-9bfb-4be5-924f-6a5596a435bf admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:11.501190 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb294dd-9bfb-4be5-924f-6a5596a435bf admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:11.557608 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260d82b2-f91f-4c4b-9e2f-398a648e8a1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:11.558740 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260d82b2-f91f-4c4b-9e2f-398a648e8a1b admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:11.559024 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260d82b2-f91f-4c4b-9e2f-398a648e8a1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:11.559298 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260d82b2-f91f-4c4b-9e2f-398a648e8a1b admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:11.559951 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260d82b2-f91f-4c4b-9e2f-398a648e8a1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:11.561618 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260d82b2-f91f-4c4b-9e2f-398a648e8a1b admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:11.571626 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1082/2184] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:32:11 2022] GET /identity/v3/roles => generated 1427 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:32:11.576578 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59dcc2a-80aa-49b1-943b-ee9ee0c5afd9 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:11.586923 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59dcc2a-80aa-49b1-943b-ee9ee0c5afd9 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:11.591659 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c75468-c5af-454d-a044-4694d401fcfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:11.592303 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c75468-c5af-454d-a044-4694d401fcfd admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:11.592584 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c75468-c5af-454d-a044-4694d401fcfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:11.592858 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c75468-c5af-454d-a044-4694d401fcfd admin admin] PATH_INFO: `/v3/projects/57bbcfd7a34d41d3b27d332c5adede46/users/4b04c3d415514ec38d4b28d19d8504e2/roles/36577a6f46d642d4b81805ede9872490` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:11.594783 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c75468-c5af-454d-a044-4694d401fcfd admin admin] RBAC: Authorizing `identity:create_grant(project_id=57bbcfd7a34d41d3b27d332c5adede46, user_id=4b04c3d415514ec38d4b28d19d8504e2, role_id=36577a6f46d642d4b81805ede9872490)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:11.597740 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c75468-c5af-454d-a044-4694d401fcfd admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:11.608246 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1103/2185] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:32:11 2022] PUT /identity/v3/projects/57bbcfd7a34d41d3b27d332c5adede46/users/4b04c3d415514ec38d4b28d19d8504e2/roles/36577a6f46d642d4b81805ede9872490 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:32:12.286181 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260d82b2-f91f-4c4b-9e2f-398a648e8a1b admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:12.288195 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-260d82b2-f91f-4c4b-9e2f-398a648e8a1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:12.305594 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260d82b2-f91f-4c4b-9e2f-398a648e8a1b admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:12.329604 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43408bbf-4ac3-44c5-87d9-23d015973707 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:12.331051 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43408bbf-4ac3-44c5-87d9-23d015973707 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:12.331495 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43408bbf-4ac3-44c5-87d9-23d015973707 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:12.403880 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-43408bbf-4ac3-44c5-87d9-23d015973707 None None] MFA Rules not processed for user `4b04c3d415514ec38d4b28d19d8504e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:32:12.407102 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-43408bbf-4ac3-44c5-87d9-23d015973707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:12.408765 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc46f52e-77db-4585-af63-8c4b8cf023ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:12.409553 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc46f52e-77db-4585-af63-8c4b8cf023ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:12.409930 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc46f52e-77db-4585-af63-8c4b8cf023ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:12.410300 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc46f52e-77db-4585-af63-8c4b8cf023ea admin admin] PATH_INFO: `/v3/users/0f83be6352184bcab5f842eb7739ec3b` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:12.411893 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc46f52e-77db-4585-af63-8c4b8cf023ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f83be6352184bcab5f842eb7739ec3b)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:12.414446 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc46f52e-77db-4585-af63-8c4b8cf023ea admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:12.461928 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:32:12.504836 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-bc46f52e-77db-4585-af63-8c4b8cf023ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f83be6352184bcab5f842eb7739ec3b'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:12.505539 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-bc46f52e-77db-4585-af63-8c4b8cf023ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f83be6352184bcab5f842eb7739ec3b'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:12.525682 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-bc46f52e-77db-4585-af63-8c4b8cf023ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f83be6352184bcab5f842eb7739ec3b'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:12.540816 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-bc46f52e-77db-4585-af63-8c4b8cf023ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f83be6352184bcab5f842eb7739ec3b'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:12.541562 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-bc46f52e-77db-4585-af63-8c4b8cf023ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f83be6352184bcab5f842eb7739ec3b'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:12.551453 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1104/2186] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:32:12 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:32:12.564162 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43408bbf-4ac3-44c5-87d9-23d015973707 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:12.565523 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1083/2187] 10.222.0.102 () {58 vars in 1229 bytes} [Thu Sep 1 17:32:12 2022] DELETE /identity/v3/users/0f83be6352184bcab5f842eb7739ec3b => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:32:12.568613 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-43408bbf-4ac3-44c5-87d9-23d015973707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:12.570150 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb8f408-3f5b-41b2-830a-d364f8f1e27e None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:12.570638 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb8f408-3f5b-41b2-830a-d364f8f1e27e None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:12.571111 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb8f408-3f5b-41b2-830a-d364f8f1e27e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:12.596331 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-4eb8f408-3f5b-41b2-830a-d364f8f1e27e None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:32:12.597589 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-4eb8f408-3f5b-41b2-830a-d364f8f1e27e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:12.622305 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:32:12.638205 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43408bbf-4ac3-44c5-87d9-23d015973707 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:12.668803 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1084/2188] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:32:12 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:32:12.678266 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb8f408-3f5b-41b2-830a-d364f8f1e27e None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:12.679632 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-4eb8f408-3f5b-41b2-830a-d364f8f1e27e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:12.695781 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4500b186-e207-43dd-9339-004e778410c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '523ed589872a45a09b6a7675a7fb0b75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:12.696325 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4500b186-e207-43dd-9339-004e778410c6 service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:12.696603 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4500b186-e207-43dd-9339-004e778410c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:12.696873 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4500b186-e207-43dd-9339-004e778410c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:12.698613 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-4500b186-e207-43dd-9339-004e778410c6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:12.740554 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb8f408-3f5b-41b2-830a-d364f8f1e27e None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:12.744644 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b6de81-6d2b-477c-964f-9c3277e2b67c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:12.745249 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b6de81-6d2b-477c-964f-9c3277e2b67c service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:12.746696 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b6de81-6d2b-477c-964f-9c3277e2b67c service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:12.746974 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b6de81-6d2b-477c-964f-9c3277e2b67c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:12.756212 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b6de81-6d2b-477c-964f-9c3277e2b67c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:12.757664 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b6de81-6d2b-477c-964f-9c3277e2b67c service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:12.760489 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4500b186-e207-43dd-9339-004e778410c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:12.762614 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4500b186-e207-43dd-9339-004e778410c6 service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:12.768188 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1085/2189] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:32:12 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 17:32:12.772344 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1105/2190] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:32:12 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 17:32:13.063750 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b6de81-6d2b-477c-964f-9c3277e2b67c service neutron] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:13.066834 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-46b6de81-6d2b-477c-964f-9c3277e2b67c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:13.098525 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b6de81-6d2b-477c-964f-9c3277e2b67c service neutron] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:13.107250 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a845e98-466e-463d-9727-150cafe3b93d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:13.108347 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a845e98-466e-463d-9727-150cafe3b93d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:13.108958 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a845e98-466e-463d-9727-150cafe3b93d admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:13.109381 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a845e98-466e-463d-9727-150cafe3b93d admin admin] PATH_INFO: `/v3/projects/5f340b6490a9427abee7ef0c60de08f2` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:13.116313 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a845e98-466e-463d-9727-150cafe3b93d admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f340b6490a9427abee7ef0c60de08f2)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:13.119397 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a845e98-466e-463d-9727-150cafe3b93d admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:13.167093 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:32:13.172482 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-2a845e98-466e-463d-9727-150cafe3b93d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f340b6490a9427abee7ef0c60de08f2', 'request_id': 'req-2a845e98-466e-463d-9727-150cafe3b93d'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:13.179898 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-2a845e98-466e-463d-9727-150cafe3b93d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f340b6490a9427abee7ef0c60de08f2', 'request_id': 'req-2a845e98-466e-463d-9727-150cafe3b93d'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:13.189625 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-2a845e98-466e-463d-9727-150cafe3b93d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f340b6490a9427abee7ef0c60de08f2', 'request_id': 'req-2a845e98-466e-463d-9727-150cafe3b93d'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:13.202138 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-2a845e98-466e-463d-9727-150cafe3b93d admin admin] The token cache is being invalidate because project 5f340b6490a9427abee7ef0c60de08f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:32:13.202434 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-2a845e98-466e-463d-9727-150cafe3b93d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:13.203151 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-2a845e98-466e-463d-9727-150cafe3b93d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:13.204864 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1086/2191] 10.222.0.102 () {58 vars in 1238 bytes} [Thu Sep 1 17:32:13 2022] DELETE /identity/v3/projects/5f340b6490a9427abee7ef0c60de08f2 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:32:13.209008 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4500b186-e207-43dd-9339-004e778410c6 service cinder] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:13.221398 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4500b186-e207-43dd-9339-004e778410c6 service cinder] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:13.285589 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d452f98-329c-42c6-90b8-faf1d1d0e131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:13.286244 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d452f98-329c-42c6-90b8-faf1d1d0e131 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:13.286575 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d452f98-329c-42c6-90b8-faf1d1d0e131 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:13.286850 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d452f98-329c-42c6-90b8-faf1d1d0e131 admin admin] PATH_INFO: `/v3/users/ffa6c849eb85444093b76fcf388d0676` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:13.287971 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d452f98-329c-42c6-90b8-faf1d1d0e131 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ffa6c849eb85444093b76fcf388d0676)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:13.289996 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d452f98-329c-42c6-90b8-faf1d1d0e131 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:13.343817 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-2d452f98-329c-42c6-90b8-faf1d1d0e131 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ffa6c849eb85444093b76fcf388d0676'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:13.351292 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:32:13.357885 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-2d452f98-329c-42c6-90b8-faf1d1d0e131 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ffa6c849eb85444093b76fcf388d0676'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:13.366787 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-2d452f98-329c-42c6-90b8-faf1d1d0e131 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ffa6c849eb85444093b76fcf388d0676'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:13.367470 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-2d452f98-329c-42c6-90b8-faf1d1d0e131 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ffa6c849eb85444093b76fcf388d0676'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:13.368080 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-2d452f98-329c-42c6-90b8-faf1d1d0e131 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ffa6c849eb85444093b76fcf388d0676'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:13.384636 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1106/2192] 10.222.0.102 () {58 vars in 1229 bytes} [Thu Sep 1 17:32:13 2022] DELETE /identity/v3/users/ffa6c849eb85444093b76fcf388d0676 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:32:13.521347 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a845e98-466e-463d-9727-150cafe3b93d admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:13.522603 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-2a845e98-466e-463d-9727-150cafe3b93d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:13.537008 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a845e98-466e-463d-9727-150cafe3b93d admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:13.615731 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5d0f72-7295-4054-8c28-6f044c016bf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:13.616385 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5d0f72-7295-4054-8c28-6f044c016bf9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:13.616662 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5d0f72-7295-4054-8c28-6f044c016bf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:13.616931 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5d0f72-7295-4054-8c28-6f044c016bf9 admin admin] PATH_INFO: `/v3/users/a3dd1402040e4796ac35015d03f44481` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:13.618234 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5d0f72-7295-4054-8c28-6f044c016bf9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3dd1402040e4796ac35015d03f44481)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:13.620157 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5d0f72-7295-4054-8c28-6f044c016bf9 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:13.697769 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-1c5d0f72-7295-4054-8c28-6f044c016bf9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3dd1402040e4796ac35015d03f44481'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:13.697769 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-1c5d0f72-7295-4054-8c28-6f044c016bf9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3dd1402040e4796ac35015d03f44481'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:13.702016 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d452f98-329c-42c6-90b8-faf1d1d0e131 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:13.702900 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-2d452f98-329c-42c6-90b8-faf1d1d0e131 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:13.724019 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d452f98-329c-42c6-90b8-faf1d1d0e131 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:13.724827 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-1c5d0f72-7295-4054-8c28-6f044c016bf9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3dd1402040e4796ac35015d03f44481'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:13.731323 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054e436a-9a45-4462-a13d-e10232f870e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:13.732541 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054e436a-9a45-4462-a13d-e10232f870e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:13.733162 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054e436a-9a45-4462-a13d-e10232f870e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:13.733797 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054e436a-9a45-4462-a13d-e10232f870e9 admin admin] PATH_INFO: `/v3/projects/f7b968ff5d004f42a053a8971a6ee361` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:13.735831 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054e436a-9a45-4462-a13d-e10232f870e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7b968ff5d004f42a053a8971a6ee361)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:13.737114 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-1c5d0f72-7295-4054-8c28-6f044c016bf9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3dd1402040e4796ac35015d03f44481'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:13.738730 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-1c5d0f72-7295-4054-8c28-6f044c016bf9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3dd1402040e4796ac35015d03f44481'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:13.739012 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054e436a-9a45-4462-a13d-e10232f870e9 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:13.754361 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1087/2193] 10.222.0.102 () {58 vars in 1229 bytes} [Thu Sep 1 17:32:13 2022] DELETE /identity/v3/users/a3dd1402040e4796ac35015d03f44481 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:32:13.760568 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4744271a-d5a9-4d2a-a8c4-637233f05517 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:13.761094 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4744271a-d5a9-4d2a-a8c4-637233f05517 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:13.762084 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4744271a-d5a9-4d2a-a8c4-637233f05517 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:13.787510 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-4744271a-d5a9-4d2a-a8c4-637233f05517 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:32:13.788803 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-4744271a-d5a9-4d2a-a8c4-637233f05517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:13.834581 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-054e436a-9a45-4462-a13d-e10232f870e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7b968ff5d004f42a053a8971a6ee361', 'request_id': 'req-054e436a-9a45-4462-a13d-e10232f870e9'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:13.844799 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-054e436a-9a45-4462-a13d-e10232f870e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7b968ff5d004f42a053a8971a6ee361', 'request_id': 'req-054e436a-9a45-4462-a13d-e10232f870e9'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:13.857705 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:32:13.859625 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-054e436a-9a45-4462-a13d-e10232f870e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7b968ff5d004f42a053a8971a6ee361', 'request_id': 'req-054e436a-9a45-4462-a13d-e10232f870e9'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:13.871960 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-054e436a-9a45-4462-a13d-e10232f870e9 admin admin] The token cache is being invalidate because project f7b968ff5d004f42a053a8971a6ee361 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:32:13.872257 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-054e436a-9a45-4462-a13d-e10232f870e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:13.872894 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-054e436a-9a45-4462-a13d-e10232f870e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:13.874674 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1107/2194] 10.222.0.102 () {58 vars in 1238 bytes} [Thu Sep 1 17:32:13 2022] DELETE /identity/v3/projects/f7b968ff5d004f42a053a8971a6ee361 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:32:13.878256 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1088/2195] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:32:13 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:32:13.879001 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e92f65-3cc6-4a13-88d3-88b24a30c10c None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:13.879288 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e92f65-3cc6-4a13-88d3-88b24a30c10c None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:13.879563 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e92f65-3cc6-4a13-88d3-88b24a30c10c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:13.891009 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4744271a-d5a9-4d2a-a8c4-637233f05517 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:13.892786 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-4744271a-d5a9-4d2a-a8c4-637233f05517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:13.901586 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-e4e92f65-3cc6-4a13-88d3-88b24a30c10c None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:32:13.906654 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-e4e92f65-3cc6-4a13-88d3-88b24a30c10c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:13.956793 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4744271a-d5a9-4d2a-a8c4-637233f05517 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:13.972834 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1108/2196] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:32:13 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:32:13.981343 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5350e9-3ccf-40d9-9215-30cf318fc4d9 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:13.981648 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5350e9-3ccf-40d9-9215-30cf318fc4d9 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:13.981919 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5350e9-3ccf-40d9-9215-30cf318fc4d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:14.001513 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-0e5350e9-3ccf-40d9-9215-30cf318fc4d9 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:32:14.002702 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-0e5350e9-3ccf-40d9-9215-30cf318fc4d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:14.007273 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4148316-f5ef-4f5b-8c1d-1104f0f94c09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:14.007878 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4148316-f5ef-4f5b-8c1d-1104f0f94c09 service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:14.008157 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4148316-f5ef-4f5b-8c1d-1104f0f94c09 service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:14.008429 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4148316-f5ef-4f5b-8c1d-1104f0f94c09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:14.010244 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-c4148316-f5ef-4f5b-8c1d-1104f0f94c09 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:14.014778 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1109/2197] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:32:13 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:32:14.015943 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5350e9-3ccf-40d9-9215-30cf318fc4d9 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:14.026509 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5350e9-3ccf-40d9-9215-30cf318fc4d9 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:14.026942 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4148316-f5ef-4f5b-8c1d-1104f0f94c09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:14.028492 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4148316-f5ef-4f5b-8c1d-1104f0f94c09 service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:14.034297 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d1f930-1625-4246-becf-31606209f436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:14.034910 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d1f930-1625-4246-becf-31606209f436 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:14.035186 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d1f930-1625-4246-becf-31606209f436 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:14.035453 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d1f930-1625-4246-becf-31606209f436 admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:14.036130 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d1f930-1625-4246-becf-31606209f436 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:32:14.036400 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d1f930-1625-4246-becf-31606209f436 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:14.038024 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d1f930-1625-4246-becf-31606209f436 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:14.040347 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1089/2198] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:32:13 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:32:14.052248 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1110/2199] 10.222.0.102 () {58 vars in 1158 bytes} [Thu Sep 1 17:32:14 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:32:14.056665 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4148316-f5ef-4f5b-8c1d-1104f0f94c09 service neutron] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:14.066278 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4148316-f5ef-4f5b-8c1d-1104f0f94c09 service neutron] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:14.070969 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65300ae-f17d-407c-9d87-7710bd1a03a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:14.071571 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65300ae-f17d-407c-9d87-7710bd1a03a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:14.071851 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65300ae-f17d-407c-9d87-7710bd1a03a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:14.072128 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65300ae-f17d-407c-9d87-7710bd1a03a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:14.072789 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65300ae-f17d-407c-9d87-7710bd1a03a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:14.074770 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65300ae-f17d-407c-9d87-7710bd1a03a7 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:14.102450 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1090/2200] 10.222.0.102 () {60 vars in 1158 bytes} [Thu Sep 1 17:32:14 2022] POST /identity/v3/projects => generated 374 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:32:14.106853 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d1f930-1625-4246-becf-31606209f436 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:14.117474 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d1f930-1625-4246-becf-31606209f436 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:14.159256 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:32:14.197717 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba7c3b7-e8e5-4673-ae23-2a641588f4eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:14.198336 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba7c3b7-e8e5-4673-ae23-2a641588f4eb admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:14.198617 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba7c3b7-e8e5-4673-ae23-2a641588f4eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:14.199582 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba7c3b7-e8e5-4673-ae23-2a641588f4eb admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:14.200209 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba7c3b7-e8e5-4673-ae23-2a641588f4eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:14.202204 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba7c3b7-e8e5-4673-ae23-2a641588f4eb admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:14.224630 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1111/2201] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:32:14 2022] POST /identity/v3/users => generated 288 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:32:14.228869 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65300ae-f17d-407c-9d87-7710bd1a03a7 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:14.241109 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65300ae-f17d-407c-9d87-7710bd1a03a7 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:14.246937 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1aaf61-fddd-4d5d-9039-9893d29207b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:14.247663 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1aaf61-fddd-4d5d-9039-9893d29207b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:14.248019 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1aaf61-fddd-4d5d-9039-9893d29207b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:14.248356 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1aaf61-fddd-4d5d-9039-9893d29207b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:14.249072 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1aaf61-fddd-4d5d-9039-9893d29207b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:14.250844 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1aaf61-fddd-4d5d-9039-9893d29207b9 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:14.263212 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1091/2202] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:32:14 2022] GET /identity/v3/roles => generated 1427 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:32:14.267866 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba7c3b7-e8e5-4673-ae23-2a641588f4eb admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:14.278717 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba7c3b7-e8e5-4673-ae23-2a641588f4eb admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:14.283630 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66eb92c-bed9-47f4-94aa-dc6425efc8f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:14.284301 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66eb92c-bed9-47f4-94aa-dc6425efc8f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:14.284664 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66eb92c-bed9-47f4-94aa-dc6425efc8f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:14.284999 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66eb92c-bed9-47f4-94aa-dc6425efc8f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:14.285757 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66eb92c-bed9-47f4-94aa-dc6425efc8f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:14.287425 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66eb92c-bed9-47f4-94aa-dc6425efc8f8 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:14.298332 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1112/2203] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:32:14 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:32:14.302621 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1aaf61-fddd-4d5d-9039-9893d29207b9 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:14.313875 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1aaf61-fddd-4d5d-9039-9893d29207b9 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:14.319236 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10469f60-cd5b-4dd5-9478-5809df3ed42d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:14.319887 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10469f60-cd5b-4dd5-9478-5809df3ed42d admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:14.320166 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10469f60-cd5b-4dd5-9478-5809df3ed42d admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:14.323109 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10469f60-cd5b-4dd5-9478-5809df3ed42d admin admin] PATH_INFO: `/v3/projects/5f5e7805e6ae4547a767b7b382a78082/users/b54b95db13d84fc4b026fdd119cb7ec5/roles/b75cb360925e4ba5a65f14965c103ffd` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:14.324860 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66eb92c-bed9-47f4-94aa-dc6425efc8f8 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:14.326623 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-a66eb92c-bed9-47f4-94aa-dc6425efc8f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:14.339602 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66eb92c-bed9-47f4-94aa-dc6425efc8f8 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:14.341231 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10469f60-cd5b-4dd5-9478-5809df3ed42d admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f5e7805e6ae4547a767b7b382a78082, user_id=b54b95db13d84fc4b026fdd119cb7ec5, role_id=b75cb360925e4ba5a65f14965c103ffd)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:14.344034 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10469f60-cd5b-4dd5-9478-5809df3ed42d admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:14.345881 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ab81ed-d9f4-4bde-af97-006333b597b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:14.347253 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ab81ed-d9f4-4bde-af97-006333b597b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:14.347705 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ab81ed-d9f4-4bde-af97-006333b597b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:14.347977 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ab81ed-d9f4-4bde-af97-006333b597b2 admin admin] PATH_INFO: `/v3/projects/2dd0527a96e948b58c3a728fd0afb070` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:14.349136 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ab81ed-d9f4-4bde-af97-006333b597b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2dd0527a96e948b58c3a728fd0afb070)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:14.351182 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ab81ed-d9f4-4bde-af97-006333b597b2 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:14.360088 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1092/2204] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:32:14 2022] PUT /identity/v3/projects/5f5e7805e6ae4547a767b7b382a78082/users/b54b95db13d84fc4b026fdd119cb7ec5/roles/b75cb360925e4ba5a65f14965c103ffd => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:32:14.365894 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555b56cd-2e3f-44fe-a561-8b87c9ac3d24 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:14.366172 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555b56cd-2e3f-44fe-a561-8b87c9ac3d24 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:14.366444 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555b56cd-2e3f-44fe-a561-8b87c9ac3d24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:14.387200 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-555b56cd-2e3f-44fe-a561-8b87c9ac3d24 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:32:14.388397 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-555b56cd-2e3f-44fe-a561-8b87c9ac3d24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:14.404954 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c7ab81ed-d9f4-4bde-af97-006333b597b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2dd0527a96e948b58c3a728fd0afb070', 'request_id': 'req-c7ab81ed-d9f4-4bde-af97-006333b597b2'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:14.415942 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c7ab81ed-d9f4-4bde-af97-006333b597b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2dd0527a96e948b58c3a728fd0afb070', 'request_id': 'req-c7ab81ed-d9f4-4bde-af97-006333b597b2'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:14.424176 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c7ab81ed-d9f4-4bde-af97-006333b597b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2dd0527a96e948b58c3a728fd0afb070', 'request_id': 'req-c7ab81ed-d9f4-4bde-af97-006333b597b2'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:14.451611 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c7ab81ed-d9f4-4bde-af97-006333b597b2 admin admin] The token cache is being invalidate because project 2dd0527a96e948b58c3a728fd0afb070 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:32:14.452038 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c7ab81ed-d9f4-4bde-af97-006333b597b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:14.452776 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c7ab81ed-d9f4-4bde-af97-006333b597b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:14.455397 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1113/2205] 10.222.0.102 () {58 vars in 1238 bytes} [Thu Sep 1 17:32:14 2022] DELETE /identity/v3/projects/2dd0527a96e948b58c3a728fd0afb070 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:32:14.460761 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b8f871-ee73-4b92-9329-3c24a859c993 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:14.461014 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b8f871-ee73-4b92-9329-3c24a859c993 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:14.461404 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b8f871-ee73-4b92-9329-3c24a859c993 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:14.474190 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:32:14.490906 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-55b8f871-ee73-4b92-9329-3c24a859c993 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:32:14.492251 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-55b8f871-ee73-4b92-9329-3c24a859c993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:14.512922 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1093/2206] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:32:14 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:32:14.526706 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555b56cd-2e3f-44fe-a561-8b87c9ac3d24 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:14.528422 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-555b56cd-2e3f-44fe-a561-8b87c9ac3d24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:14.596689 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1114/2207] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:32:14 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:32:14.599910 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d147489-d2d0-4ea6-87a5-b922e1edb482 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:14.600278 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d147489-d2d0-4ea6-87a5-b922e1edb482 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:14.600610 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d147489-d2d0-4ea6-87a5-b922e1edb482 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:14.632744 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-7d147489-d2d0-4ea6-87a5-b922e1edb482 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:32:14.633998 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-7d147489-d2d0-4ea6-87a5-b922e1edb482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:14.646247 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1115/2208] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:32:14 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:32:14.650144 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d147489-d2d0-4ea6-87a5-b922e1edb482 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:14.659872 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d147489-d2d0-4ea6-87a5-b922e1edb482 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:14.660516 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555b56cd-2e3f-44fe-a561-8b87c9ac3d24 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:14.664403 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabb40eb-6ccf-4fd7-ab0b-5b5d04ed2808 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:14.665002 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabb40eb-6ccf-4fd7-ab0b-5b5d04ed2808 service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:14.665278 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabb40eb-6ccf-4fd7-ab0b-5b5d04ed2808 service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:14.665443 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf25f16-8a99-4a0f-ae1d-96b3d1e2c07e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:14.665619 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabb40eb-6ccf-4fd7-ab0b-5b5d04ed2808 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:14.666365 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf25f16-8a99-4a0f-ae1d-96b3d1e2c07e admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:14.666652 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf25f16-8a99-4a0f-ae1d-96b3d1e2c07e admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:14.666951 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf25f16-8a99-4a0f-ae1d-96b3d1e2c07e admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:14.667587 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf25f16-8a99-4a0f-ae1d-96b3d1e2c07e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:32:14.668411 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf25f16-8a99-4a0f-ae1d-96b3d1e2c07e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:14.670032 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf25f16-8a99-4a0f-ae1d-96b3d1e2c07e admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:14.678180 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabb40eb-6ccf-4fd7-ab0b-5b5d04ed2808 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:14.680272 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabb40eb-6ccf-4fd7-ab0b-5b5d04ed2808 service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:14.682724 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1116/2209] 10.222.0.102 () {58 vars in 1158 bytes} [Thu Sep 1 17:32:14 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:32:14.688523 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf25f16-8a99-4a0f-ae1d-96b3d1e2c07e admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:14.694495 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1094/2210] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:32:14 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:32:14.702756 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf25f16-8a99-4a0f-ae1d-96b3d1e2c07e admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:14.708860 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0299bd-c72c-4271-9de0-3536c9731502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:14.709563 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0299bd-c72c-4271-9de0-3536c9731502 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:14.709843 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0299bd-c72c-4271-9de0-3536c9731502 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:14.710127 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0299bd-c72c-4271-9de0-3536c9731502 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:14.710759 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0299bd-c72c-4271-9de0-3536c9731502 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:14.714441 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0299bd-c72c-4271-9de0-3536c9731502 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:14.750786 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:32:14.762640 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1117/2211] 10.222.0.102 () {60 vars in 1158 bytes} [Thu Sep 1 17:32:14 2022] POST /identity/v3/projects => generated 372 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:32:14.767193 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabb40eb-6ccf-4fd7-ab0b-5b5d04ed2808 service neutron] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:14.780790 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabb40eb-6ccf-4fd7-ab0b-5b5d04ed2808 service neutron] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:14.878167 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acea224-ed3b-464a-bce0-b9ffc5bf5454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:14.878783 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acea224-ed3b-464a-bce0-b9ffc5bf5454 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:14.879150 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acea224-ed3b-464a-bce0-b9ffc5bf5454 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:14.879430 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acea224-ed3b-464a-bce0-b9ffc5bf5454 admin admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:14.880066 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acea224-ed3b-464a-bce0-b9ffc5bf5454 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:14.882027 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acea224-ed3b-464a-bce0-b9ffc5bf5454 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:14.918577 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1095/2212] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:32:14 2022] POST /identity/v3/users => generated 287 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:32:14.918933 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0299bd-c72c-4271-9de0-3536c9731502 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:14.936407 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0299bd-c72c-4271-9de0-3536c9731502 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:14.953471 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51af6c05-fa51-4ca2-bb5a-03b45960a52b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:14.954190 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51af6c05-fa51-4ca2-bb5a-03b45960a52b admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:14.954518 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51af6c05-fa51-4ca2-bb5a-03b45960a52b admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:14.954832 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51af6c05-fa51-4ca2-bb5a-03b45960a52b admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:14.955520 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51af6c05-fa51-4ca2-bb5a-03b45960a52b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:14.957382 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51af6c05-fa51-4ca2-bb5a-03b45960a52b admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:14.971521 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1118/2213] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:32:14 2022] GET /identity/v3/roles => generated 1427 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:32:14.974641 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acea224-ed3b-464a-bce0-b9ffc5bf5454 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:14.986013 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acea224-ed3b-464a-bce0-b9ffc5bf5454 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:14.990969 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6df73d-83bb-44c4-9ed5-8890118821b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:14.991579 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6df73d-83bb-44c4-9ed5-8890118821b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:14.991863 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6df73d-83bb-44c4-9ed5-8890118821b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:14.992140 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6df73d-83bb-44c4-9ed5-8890118821b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:14.992788 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6df73d-83bb-44c4-9ed5-8890118821b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:14.994435 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6df73d-83bb-44c4-9ed5-8890118821b2 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:14.997245 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51af6c05-fa51-4ca2-bb5a-03b45960a52b admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:14.999634 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-51af6c05-fa51-4ca2-bb5a-03b45960a52b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:15.013207 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51af6c05-fa51-4ca2-bb5a-03b45960a52b admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:15.014769 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1096/2214] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:32:14 2022] GET /identity/v3/roles => generated 1427 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:32:15.019732 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6df73d-83bb-44c4-9ed5-8890118821b2 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:15.021793 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad36ea3-82bd-4b2c-8cde-21995ba2817e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:15.022560 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad36ea3-82bd-4b2c-8cde-21995ba2817e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:15.022917 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad36ea3-82bd-4b2c-8cde-21995ba2817e admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:15.023262 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad36ea3-82bd-4b2c-8cde-21995ba2817e admin admin] PATH_INFO: `/v3/users/fb82355daa6a4346ab93fb058788c1fb` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:15.024560 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad36ea3-82bd-4b2c-8cde-21995ba2817e admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb82355daa6a4346ab93fb058788c1fb)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:15.028453 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad36ea3-82bd-4b2c-8cde-21995ba2817e admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:15.034717 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6df73d-83bb-44c4-9ed5-8890118821b2 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:15.040027 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aac5627-a9bb-46cd-b56f-f5218d3751ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:15.040781 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aac5627-a9bb-46cd-b56f-f5218d3751ad admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:15.041127 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aac5627-a9bb-46cd-b56f-f5218d3751ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:15.041494 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aac5627-a9bb-46cd-b56f-f5218d3751ad admin admin] PATH_INFO: `/v3/projects/a3dcb0d838e64eb7ab328c36599f3eaf/users/868c916f3de045d5860b814999d8e93c/roles/b75cb360925e4ba5a65f14965c103ffd` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:15.077223 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aac5627-a9bb-46cd-b56f-f5218d3751ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3dcb0d838e64eb7ab328c36599f3eaf, user_id=868c916f3de045d5860b814999d8e93c, role_id=b75cb360925e4ba5a65f14965c103ffd)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:15.081770 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aac5627-a9bb-46cd-b56f-f5218d3751ad admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:15.099432 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1097/2215] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:32:15 2022] PUT /identity/v3/projects/a3dcb0d838e64eb7ab328c36599f3eaf/users/868c916f3de045d5860b814999d8e93c/roles/b75cb360925e4ba5a65f14965c103ffd => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:32:15.111246 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae400c3b-684a-43bf-a640-994e309a2f4e None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:15.111246 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae400c3b-684a-43bf-a640-994e309a2f4e None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:15.111711 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae400c3b-684a-43bf-a640-994e309a2f4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:15.115052 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-4ad36ea3-82bd-4b2c-8cde-21995ba2817e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb82355daa6a4346ab93fb058788c1fb'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:15.131664 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-4ad36ea3-82bd-4b2c-8cde-21995ba2817e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb82355daa6a4346ab93fb058788c1fb'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:15.140897 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-4ad36ea3-82bd-4b2c-8cde-21995ba2817e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb82355daa6a4346ab93fb058788c1fb'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:15.141668 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-4ad36ea3-82bd-4b2c-8cde-21995ba2817e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb82355daa6a4346ab93fb058788c1fb'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:15.142387 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-4ad36ea3-82bd-4b2c-8cde-21995ba2817e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb82355daa6a4346ab93fb058788c1fb'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:15.149602 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-ae400c3b-684a-43bf-a640-994e309a2f4e None None] MFA Rules not processed for user `868c916f3de045d5860b814999d8e93c`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:32:15.152074 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-ae400c3b-684a-43bf-a640-994e309a2f4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:15.157180 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1119/2216] 10.222.0.102 () {58 vars in 1229 bytes} [Thu Sep 1 17:32:14 2022] DELETE /identity/v3/users/fb82355daa6a4346ab93fb058788c1fb => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:32:15.162516 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec406e70-1458-4424-a98a-2e91bcd9dac7 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:15.162971 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec406e70-1458-4424-a98a-2e91bcd9dac7 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:15.163386 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec406e70-1458-4424-a98a-2e91bcd9dac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:15.186599 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-ec406e70-1458-4424-a98a-2e91bcd9dac7 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:32:15.188127 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-ec406e70-1458-4424-a98a-2e91bcd9dac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:15.246653 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:32:15.278601 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1120/2217] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:32:15 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:32:15.294190 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec406e70-1458-4424-a98a-2e91bcd9dac7 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:15.299261 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-ec406e70-1458-4424-a98a-2e91bcd9dac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:15.300168 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1098/2218] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:32:15 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:32:15.308867 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae400c3b-684a-43bf-a640-994e309a2f4e None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:15.311104 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-ae400c3b-684a-43bf-a640-994e309a2f4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:15.377127 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae400c3b-684a-43bf-a640-994e309a2f4e None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:15.380708 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec406e70-1458-4424-a98a-2e91bcd9dac7 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:15.381610 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f702e479-1310-4e6b-9257-812fa86d6e8a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '523ed589872a45a09b6a7675a7fb0b75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:15.382255 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f702e479-1310-4e6b-9257-812fa86d6e8a service cinder] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:15.382951 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f702e479-1310-4e6b-9257-812fa86d6e8a service cinder] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:15.383398 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f702e479-1310-4e6b-9257-812fa86d6e8a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:15.385602 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-f702e479-1310-4e6b-9257-812fa86d6e8a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:15.386217 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d0b053-5579-4a86-8f90-1e8161104e93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:15.387190 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d0b053-5579-4a86-8f90-1e8161104e93 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:15.387476 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d0b053-5579-4a86-8f90-1e8161104e93 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:15.387904 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d0b053-5579-4a86-8f90-1e8161104e93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:15.397894 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d0b053-5579-4a86-8f90-1e8161104e93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:15.399359 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d0b053-5579-4a86-8f90-1e8161104e93 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:15.411228 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1121/2219] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:32:15 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:32:15.413523 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d0b053-5579-4a86-8f90-1e8161104e93 service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:15.415299 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-60d0b053-5579-4a86-8f90-1e8161104e93 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:15.429552 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:32:15.441946 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d0b053-5579-4a86-8f90-1e8161104e93 service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:15.447745 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a76b2a-60a6-4202-a030-171e2082592b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '523ed589872a45a09b6a7675a7fb0b75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:15.449565 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a76b2a-60a6-4202-a030-171e2082592b service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:15.450024 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a76b2a-60a6-4202-a030-171e2082592b service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:15.450575 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a76b2a-60a6-4202-a030-171e2082592b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:15.463646 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a76b2a-60a6-4202-a030-171e2082592b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:15.466425 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a76b2a-60a6-4202-a030-171e2082592b service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:15.475582 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f702e479-1310-4e6b-9257-812fa86d6e8a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:15.477074 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f702e479-1310-4e6b-9257-812fa86d6e8a service cinder] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:15.485772 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1122/2220] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:32:15 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:32:15.504968 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1099/2221] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:32:15 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:32:15.521660 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a76b2a-60a6-4202-a030-171e2082592b service cinder] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:15.531208 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-53a76b2a-60a6-4202-a030-171e2082592b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:15.558105 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a76b2a-60a6-4202-a030-171e2082592b service cinder] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:15.565735 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736b8795-1d81-479d-8a21-2e90b5389ac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:15.568247 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff455c67-3e9f-4d23-8003-373f5e9e1139 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:15.568721 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff455c67-3e9f-4d23-8003-373f5e9e1139 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:15.568959 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff455c67-3e9f-4d23-8003-373f5e9e1139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:15.576009 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736b8795-1d81-479d-8a21-2e90b5389ac4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:15.576289 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736b8795-1d81-479d-8a21-2e90b5389ac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:15.579521 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736b8795-1d81-479d-8a21-2e90b5389ac4 admin admin] PATH_INFO: `/v3/users/868c916f3de045d5860b814999d8e93c` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:15.581627 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736b8795-1d81-479d-8a21-2e90b5389ac4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=868c916f3de045d5860b814999d8e93c)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:15.587694 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736b8795-1d81-479d-8a21-2e90b5389ac4 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:15.619460 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-ff455c67-3e9f-4d23-8003-373f5e9e1139 None None] MFA Rules not processed for user `b54b95db13d84fc4b026fdd119cb7ec5`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:32:15.621698 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-ff455c67-3e9f-4d23-8003-373f5e9e1139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:15.711763 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-736b8795-1d81-479d-8a21-2e90b5389ac4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '868c916f3de045d5860b814999d8e93c'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:15.757759 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-736b8795-1d81-479d-8a21-2e90b5389ac4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '868c916f3de045d5860b814999d8e93c'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:15.763314 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-736b8795-1d81-479d-8a21-2e90b5389ac4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '868c916f3de045d5860b814999d8e93c'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:15.764700 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-736b8795-1d81-479d-8a21-2e90b5389ac4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '868c916f3de045d5860b814999d8e93c'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:15.766912 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-736b8795-1d81-479d-8a21-2e90b5389ac4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '868c916f3de045d5860b814999d8e93c'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:15.809325 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1123/2222] 10.222.0.102 () {58 vars in 1229 bytes} [Thu Sep 1 17:32:15 2022] DELETE /identity/v3/users/868c916f3de045d5860b814999d8e93c => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:32:15.822547 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a86f8b-7881-4486-a22d-119936351957 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:15.822965 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a86f8b-7881-4486-a22d-119936351957 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:15.823343 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a86f8b-7881-4486-a22d-119936351957 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:15.824189 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1100/2223] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:32:15 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:32:15.852425 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-54a86f8b-7881-4486-a22d-119936351957 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:32:15.854246 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-54a86f8b-7881-4486-a22d-119936351957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:15.951970 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff455c67-3e9f-4d23-8003-373f5e9e1139 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:15.953415 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-ff455c67-3e9f-4d23-8003-373f5e9e1139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:15.956906 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1124/2224] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:32:15 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:32:15.973596 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a86f8b-7881-4486-a22d-119936351957 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:15.976299 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-54a86f8b-7881-4486-a22d-119936351957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:15.977346 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff455c67-3e9f-4d23-8003-373f5e9e1139 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:16.059124 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:32:16.127438 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b205041-2aa9-4ce3-af54-94cf503400d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '523ed589872a45a09b6a7675a7fb0b75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:16.129504 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b205041-2aa9-4ce3-af54-94cf503400d5 service cinder] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:16.129853 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b205041-2aa9-4ce3-af54-94cf503400d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:16.130189 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b205041-2aa9-4ce3-af54-94cf503400d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:16.168686 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a86f8b-7881-4486-a22d-119936351957 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:16.181123 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5612c2-11ae-4866-8db9-65438f81fc16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:16.181778 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5612c2-11ae-4866-8db9-65438f81fc16 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:16.182057 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5612c2-11ae-4866-8db9-65438f81fc16 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:16.182327 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5612c2-11ae-4866-8db9-65438f81fc16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:16.203983 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5612c2-11ae-4866-8db9-65438f81fc16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:16.205602 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5612c2-11ae-4866-8db9-65438f81fc16 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:16.237584 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1125/2225] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:32:15 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 17:32:16.238321 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5612c2-11ae-4866-8db9-65438f81fc16 service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:16.246645 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-6b5612c2-11ae-4866-8db9-65438f81fc16 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:16.271043 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:32:16.277019 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5612c2-11ae-4866-8db9-65438f81fc16 service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:16.286988 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b205041-2aa9-4ce3-af54-94cf503400d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:16.288192 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b205041-2aa9-4ce3-af54-94cf503400d5 service cinder] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:16.289388 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df89854-dc63-4a1d-84aa-93ed3af48f66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:16.290305 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df89854-dc63-4a1d-84aa-93ed3af48f66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:16.290692 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df89854-dc63-4a1d-84aa-93ed3af48f66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:16.290970 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df89854-dc63-4a1d-84aa-93ed3af48f66 admin admin] PATH_INFO: `/v3/projects/7514891da2db401cbd983bfc04b8da6c` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:16.293689 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df89854-dc63-4a1d-84aa-93ed3af48f66 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7514891da2db401cbd983bfc04b8da6c)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:16.295797 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df89854-dc63-4a1d-84aa-93ed3af48f66 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:16.303686 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1101/2226] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:32:15 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 17:32:16.364678 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-0df89854-dc63-4a1d-84aa-93ed3af48f66 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7514891da2db401cbd983bfc04b8da6c', 'request_id': 'req-0df89854-dc63-4a1d-84aa-93ed3af48f66'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:16.374210 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-0df89854-dc63-4a1d-84aa-93ed3af48f66 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7514891da2db401cbd983bfc04b8da6c', 'request_id': 'req-0df89854-dc63-4a1d-84aa-93ed3af48f66'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:16.382865 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-0df89854-dc63-4a1d-84aa-93ed3af48f66 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7514891da2db401cbd983bfc04b8da6c', 'request_id': 'req-0df89854-dc63-4a1d-84aa-93ed3af48f66'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:16.399924 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-0df89854-dc63-4a1d-84aa-93ed3af48f66 admin admin] The token cache is being invalidate because project 7514891da2db401cbd983bfc04b8da6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:32:16.399924 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-0df89854-dc63-4a1d-84aa-93ed3af48f66 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:16.399924 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-0df89854-dc63-4a1d-84aa-93ed3af48f66 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:16.405400 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1126/2227] 10.222.0.102 () {58 vars in 1238 bytes} [Thu Sep 1 17:32:16 2022] DELETE /identity/v3/projects/7514891da2db401cbd983bfc04b8da6c => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:32:16.407359 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b205041-2aa9-4ce3-af54-94cf503400d5 service cinder] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:16.427307 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b205041-2aa9-4ce3-af54-94cf503400d5 service cinder] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:16.518809 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46b8bb0-cd53-4f77-93ae-fdb733f4f9f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:16.519491 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46b8bb0-cd53-4f77-93ae-fdb733f4f9f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:16.519845 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46b8bb0-cd53-4f77-93ae-fdb733f4f9f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:16.520218 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46b8bb0-cd53-4f77-93ae-fdb733f4f9f3 admin admin] PATH_INFO: `/v3/users/f39f2e3c85fc44a5b20e4573256c7da9` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:16.522365 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46b8bb0-cd53-4f77-93ae-fdb733f4f9f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f39f2e3c85fc44a5b20e4573256c7da9)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:16.528055 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46b8bb0-cd53-4f77-93ae-fdb733f4f9f3 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:16.594971 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-f46b8bb0-cd53-4f77-93ae-fdb733f4f9f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f39f2e3c85fc44a5b20e4573256c7da9'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:16.596565 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-f46b8bb0-cd53-4f77-93ae-fdb733f4f9f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f39f2e3c85fc44a5b20e4573256c7da9'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:16.613145 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-f46b8bb0-cd53-4f77-93ae-fdb733f4f9f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f39f2e3c85fc44a5b20e4573256c7da9'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:16.624830 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-f46b8bb0-cd53-4f77-93ae-fdb733f4f9f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f39f2e3c85fc44a5b20e4573256c7da9'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:16.627326 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-f46b8bb0-cd53-4f77-93ae-fdb733f4f9f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f39f2e3c85fc44a5b20e4573256c7da9'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:16.632922 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:32:16.663121 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1102/2228] 10.222.0.102 () {58 vars in 1229 bytes} [Thu Sep 1 17:32:16 2022] DELETE /identity/v3/users/f39f2e3c85fc44a5b20e4573256c7da9 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:32:17.487701 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df89854-dc63-4a1d-84aa-93ed3af48f66 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:17.489235 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-0df89854-dc63-4a1d-84aa-93ed3af48f66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:17.503706 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df89854-dc63-4a1d-84aa-93ed3af48f66 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:17.572352 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf9d91c-d110-4c69-8dca-f508791fd1b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:17.573033 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf9d91c-d110-4c69-8dca-f508791fd1b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:17.573341 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf9d91c-d110-4c69-8dca-f508791fd1b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:17.573619 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf9d91c-d110-4c69-8dca-f508791fd1b2 admin admin] PATH_INFO: `/v3/projects/a3dcb0d838e64eb7ab328c36599f3eaf` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:17.575646 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf9d91c-d110-4c69-8dca-f508791fd1b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3dcb0d838e64eb7ab328c36599f3eaf)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:17.578101 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf9d91c-d110-4c69-8dca-f508791fd1b2 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:17.605524 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46b8bb0-cd53-4f77-93ae-fdb733f4f9f3 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:17.607751 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-f46b8bb0-cd53-4f77-93ae-fdb733f4f9f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:17.647712 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46b8bb0-cd53-4f77-93ae-fdb733f4f9f3 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:17.666172 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9cf9d91c-d110-4c69-8dca-f508791fd1b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3dcb0d838e64eb7ab328c36599f3eaf', 'request_id': 'req-9cf9d91c-d110-4c69-8dca-f508791fd1b2'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:17.676147 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9cf9d91c-d110-4c69-8dca-f508791fd1b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3dcb0d838e64eb7ab328c36599f3eaf', 'request_id': 'req-9cf9d91c-d110-4c69-8dca-f508791fd1b2'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:17.684572 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9cf9d91c-d110-4c69-8dca-f508791fd1b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3dcb0d838e64eb7ab328c36599f3eaf', 'request_id': 'req-9cf9d91c-d110-4c69-8dca-f508791fd1b2'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:17.701189 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9cf9d91c-d110-4c69-8dca-f508791fd1b2 admin admin] The token cache is being invalidate because project a3dcb0d838e64eb7ab328c36599f3eaf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:32:17.701524 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9cf9d91c-d110-4c69-8dca-f508791fd1b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:17.702295 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9cf9d91c-d110-4c69-8dca-f508791fd1b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:17.703975 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1127/2229] 10.222.0.102 () {58 vars in 1238 bytes} [Thu Sep 1 17:32:17 2022] DELETE /identity/v3/projects/a3dcb0d838e64eb7ab328c36599f3eaf => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:32:17.752669 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59122ac-8a20-45e2-a4d8-202ea5c71cd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:17.753472 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59122ac-8a20-45e2-a4d8-202ea5c71cd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:17.753831 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59122ac-8a20-45e2-a4d8-202ea5c71cd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:17.754355 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59122ac-8a20-45e2-a4d8-202ea5c71cd2 admin admin] PATH_INFO: `/v3/projects/eb297cdd28bb40ecb134a655266a36cf` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:17.755662 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59122ac-8a20-45e2-a4d8-202ea5c71cd2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb297cdd28bb40ecb134a655266a36cf)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:17.757979 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59122ac-8a20-45e2-a4d8-202ea5c71cd2 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:17.853596 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-b59122ac-8a20-45e2-a4d8-202ea5c71cd2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb297cdd28bb40ecb134a655266a36cf', 'request_id': 'req-b59122ac-8a20-45e2-a4d8-202ea5c71cd2'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:17.860887 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-b59122ac-8a20-45e2-a4d8-202ea5c71cd2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb297cdd28bb40ecb134a655266a36cf', 'request_id': 'req-b59122ac-8a20-45e2-a4d8-202ea5c71cd2'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:17.871928 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-b59122ac-8a20-45e2-a4d8-202ea5c71cd2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb297cdd28bb40ecb134a655266a36cf', 'request_id': 'req-b59122ac-8a20-45e2-a4d8-202ea5c71cd2'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:17.887221 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-b59122ac-8a20-45e2-a4d8-202ea5c71cd2 admin admin] The token cache is being invalidate because project eb297cdd28bb40ecb134a655266a36cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:32:17.887221 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-b59122ac-8a20-45e2-a4d8-202ea5c71cd2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:17.887922 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-b59122ac-8a20-45e2-a4d8-202ea5c71cd2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:17.892903 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1103/2230] 10.222.0.102 () {58 vars in 1238 bytes} [Thu Sep 1 17:32:17 2022] DELETE /identity/v3/projects/eb297cdd28bb40ecb134a655266a36cf => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:32:17.899231 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf9d91c-d110-4c69-8dca-f508791fd1b2 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:17.903366 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-9cf9d91c-d110-4c69-8dca-f508791fd1b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:17.920130 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf9d91c-d110-4c69-8dca-f508791fd1b2 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:17.935688 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:32:17.994809 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4209caad-66d5-42d1-942c-671a98a05ba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:17.996815 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4209caad-66d5-42d1-942c-671a98a05ba9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:17.997123 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4209caad-66d5-42d1-942c-671a98a05ba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:17.999136 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4209caad-66d5-42d1-942c-671a98a05ba9 admin admin] PATH_INFO: `/v3/users/f008a95a41164ef682b28212d8f64363` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:18.000373 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4209caad-66d5-42d1-942c-671a98a05ba9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f008a95a41164ef682b28212d8f64363)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:18.002479 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4209caad-66d5-42d1-942c-671a98a05ba9 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:18.067048 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-4209caad-66d5-42d1-942c-671a98a05ba9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f008a95a41164ef682b28212d8f64363'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:18.081339 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-4209caad-66d5-42d1-942c-671a98a05ba9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f008a95a41164ef682b28212d8f64363'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:18.090816 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-4209caad-66d5-42d1-942c-671a98a05ba9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f008a95a41164ef682b28212d8f64363'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:18.091434 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-4209caad-66d5-42d1-942c-671a98a05ba9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f008a95a41164ef682b28212d8f64363'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:18.092040 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-4209caad-66d5-42d1-942c-671a98a05ba9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f008a95a41164ef682b28212d8f64363'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:18.105369 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1128/2231] 10.222.0.102 () {58 vars in 1229 bytes} [Thu Sep 1 17:32:17 2022] DELETE /identity/v3/users/f008a95a41164ef682b28212d8f64363 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:32:18.410796 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59122ac-8a20-45e2-a4d8-202ea5c71cd2 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:18.412301 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-b59122ac-8a20-45e2-a4d8-202ea5c71cd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:18.426197 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59122ac-8a20-45e2-a4d8-202ea5c71cd2 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:18.548463 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705b3ed8-c835-4afe-bc0b-2dd45861265d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:18.549143 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705b3ed8-c835-4afe-bc0b-2dd45861265d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:18.549625 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705b3ed8-c835-4afe-bc0b-2dd45861265d admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:18.549908 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705b3ed8-c835-4afe-bc0b-2dd45861265d admin admin] PATH_INFO: `/v3/projects/ed68084e3b854558930c4b6f11ea07c7` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:18.551659 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705b3ed8-c835-4afe-bc0b-2dd45861265d admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed68084e3b854558930c4b6f11ea07c7)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:18.552964 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705b3ed8-c835-4afe-bc0b-2dd45861265d admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:18.598813 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:32:18.624572 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-705b3ed8-c835-4afe-bc0b-2dd45861265d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed68084e3b854558930c4b6f11ea07c7', 'request_id': 'req-705b3ed8-c835-4afe-bc0b-2dd45861265d'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:18.632845 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-705b3ed8-c835-4afe-bc0b-2dd45861265d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed68084e3b854558930c4b6f11ea07c7', 'request_id': 'req-705b3ed8-c835-4afe-bc0b-2dd45861265d'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:18.643363 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-705b3ed8-c835-4afe-bc0b-2dd45861265d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed68084e3b854558930c4b6f11ea07c7', 'request_id': 'req-705b3ed8-c835-4afe-bc0b-2dd45861265d'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:18.658900 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-705b3ed8-c835-4afe-bc0b-2dd45861265d admin admin] The token cache is being invalidate because project ed68084e3b854558930c4b6f11ea07c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:32:18.659207 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-705b3ed8-c835-4afe-bc0b-2dd45861265d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:18.659886 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-705b3ed8-c835-4afe-bc0b-2dd45861265d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:18.661691 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1104/2232] 10.222.0.102 () {58 vars in 1238 bytes} [Thu Sep 1 17:32:18 2022] DELETE /identity/v3/projects/ed68084e3b854558930c4b6f11ea07c7 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:32:18.668851 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787e2279-5eb2-4a6a-b27d-17182a67bf31 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:18.669392 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787e2279-5eb2-4a6a-b27d-17182a67bf31 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:18.669470 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787e2279-5eb2-4a6a-b27d-17182a67bf31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:18.691255 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-787e2279-5eb2-4a6a-b27d-17182a67bf31 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:32:18.692492 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-787e2279-5eb2-4a6a-b27d-17182a67bf31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:18.753569 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1129/2233] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:32:18 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:32:18.767629 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182f057f-cf4f-44ef-b38a-5aa89f3edc11 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:18.767928 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182f057f-cf4f-44ef-b38a-5aa89f3edc11 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:18.768191 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182f057f-cf4f-44ef-b38a-5aa89f3edc11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:18.788396 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-182f057f-cf4f-44ef-b38a-5aa89f3edc11 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:32:18.789652 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-182f057f-cf4f-44ef-b38a-5aa89f3edc11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:18.798853 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1105/2234] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:32:18 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:32:18.803326 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787e2279-5eb2-4a6a-b27d-17182a67bf31 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:18.813436 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787e2279-5eb2-4a6a-b27d-17182a67bf31 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:18.818403 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9669353-1892-48c9-8f0b-f637313b172d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:18.819010 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9669353-1892-48c9-8f0b-f637313b172d admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:18.819295 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9669353-1892-48c9-8f0b-f637313b172d admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:18.819568 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9669353-1892-48c9-8f0b-f637313b172d admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:18.820218 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9669353-1892-48c9-8f0b-f637313b172d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:32:18.820542 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9669353-1892-48c9-8f0b-f637313b172d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:18.822270 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9669353-1892-48c9-8f0b-f637313b172d admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:18.837938 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1130/2235] 10.222.0.102 () {58 vars in 1158 bytes} [Thu Sep 1 17:32:18 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:32:18.839038 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182f057f-cf4f-44ef-b38a-5aa89f3edc11 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:18.849181 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182f057f-cf4f-44ef-b38a-5aa89f3edc11 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:18.861940 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c2096c-dc33-4d73-93fb-2db771d7d9a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:18.862661 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c2096c-dc33-4d73-93fb-2db771d7d9a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:18.863013 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c2096c-dc33-4d73-93fb-2db771d7d9a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:18.863296 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c2096c-dc33-4d73-93fb-2db771d7d9a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:18.864158 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c2096c-dc33-4d73-93fb-2db771d7d9a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:18.866750 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c2096c-dc33-4d73-93fb-2db771d7d9a7 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:18.904008 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1106/2236] 10.222.0.102 () {60 vars in 1158 bytes} [Thu Sep 1 17:32:18 2022] POST /identity/v3/projects => generated 376 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:32:18.905290 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9669353-1892-48c9-8f0b-f637313b172d admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:18.916024 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9669353-1892-48c9-8f0b-f637313b172d admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:18.942496 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:32:19.005113 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34d2622-a06c-47e9-9704-2e35809938f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:19.006024 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34d2622-a06c-47e9-9704-2e35809938f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:19.006399 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34d2622-a06c-47e9-9704-2e35809938f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:19.006726 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34d2622-a06c-47e9-9704-2e35809938f1 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:19.007783 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34d2622-a06c-47e9-9704-2e35809938f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:19.010687 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34d2622-a06c-47e9-9704-2e35809938f1 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:19.050032 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1131/2237] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:32:18 2022] POST /identity/v3/users => generated 289 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:32:19.052418 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c2096c-dc33-4d73-93fb-2db771d7d9a7 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:19.064419 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c2096c-dc33-4d73-93fb-2db771d7d9a7 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:19.070814 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb7a2cb-f456-46ff-820a-0b51c60c9b6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:19.071560 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb7a2cb-f456-46ff-820a-0b51c60c9b6a admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:19.071844 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb7a2cb-f456-46ff-820a-0b51c60c9b6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:19.072124 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb7a2cb-f456-46ff-820a-0b51c60c9b6a admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:19.072820 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb7a2cb-f456-46ff-820a-0b51c60c9b6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:19.075064 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb7a2cb-f456-46ff-820a-0b51c60c9b6a admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:19.089696 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1107/2238] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:32:19 2022] GET /identity/v3/roles => generated 1427 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:32:19.099345 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34d2622-a06c-47e9-9704-2e35809938f1 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:19.108135 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26529770-73fd-4c17-a651-4e1e0dcf413a None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:19.108549 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26529770-73fd-4c17-a651-4e1e0dcf413a None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:19.109579 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26529770-73fd-4c17-a651-4e1e0dcf413a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:19.114303 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34d2622-a06c-47e9-9704-2e35809938f1 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:19.121823 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7b810e-1158-4c62-8433-e7c25bfe394e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:19.122604 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7b810e-1158-4c62-8433-e7c25bfe394e admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:19.123039 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7b810e-1158-4c62-8433-e7c25bfe394e admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:19.123312 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7b810e-1158-4c62-8433-e7c25bfe394e admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:19.123956 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7b810e-1158-4c62-8433-e7c25bfe394e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:19.125724 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7b810e-1158-4c62-8433-e7c25bfe394e admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:19.135025 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-26529770-73fd-4c17-a651-4e1e0dcf413a None None] MFA Rules not processed for user `b54b95db13d84fc4b026fdd119cb7ec5`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:32:19.136638 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-26529770-73fd-4c17-a651-4e1e0dcf413a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:19.137084 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1132/2239] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:32:19 2022] GET /identity/v3/roles => generated 1427 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:32:19.153513 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7b810e-1158-4c62-8433-e7c25bfe394e admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:19.153513 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7b810e-1158-4c62-8433-e7c25bfe394e admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:19.157444 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a534d218-b788-4c89-98fc-f6413aa3f97c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:19.158206 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a534d218-b788-4c89-98fc-f6413aa3f97c admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:19.158377 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a534d218-b788-4c89-98fc-f6413aa3f97c admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:19.158648 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a534d218-b788-4c89-98fc-f6413aa3f97c admin admin] PATH_INFO: `/v3/projects/e5d2bf10be5a47bba216ed775f0837e8/users/31cab3856ca041db82d1caf5aeeaadd0/roles/b75cb360925e4ba5a65f14965c103ffd` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:19.173969 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a534d218-b788-4c89-98fc-f6413aa3f97c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5d2bf10be5a47bba216ed775f0837e8, user_id=31cab3856ca041db82d1caf5aeeaadd0, role_id=b75cb360925e4ba5a65f14965c103ffd)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:19.176605 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a534d218-b788-4c89-98fc-f6413aa3f97c admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:19.191717 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1133/2240] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:32:19 2022] PUT /identity/v3/projects/e5d2bf10be5a47bba216ed775f0837e8/users/31cab3856ca041db82d1caf5aeeaadd0/roles/b75cb360925e4ba5a65f14965c103ffd => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:32:19.204278 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1108/2241] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:32:19 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:32:19.204446 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb64939-f05b-44e2-a66b-b172d7ce0576 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:19.204446 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb64939-f05b-44e2-a66b-b172d7ce0576 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:19.204446 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb64939-f05b-44e2-a66b-b172d7ce0576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:19.236100 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-0fb64939-f05b-44e2-a66b-b172d7ce0576 None None] MFA Rules not processed for user `31cab3856ca041db82d1caf5aeeaadd0`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:32:19.237995 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-0fb64939-f05b-44e2-a66b-b172d7ce0576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:19.315176 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1134/2242] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:32:19 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:32:19.326016 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26529770-73fd-4c17-a651-4e1e0dcf413a None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:19.327640 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-26529770-73fd-4c17-a651-4e1e0dcf413a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:19.394693 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26529770-73fd-4c17-a651-4e1e0dcf413a None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:19.433982 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:32:19.454889 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9da305-483e-4489-9343-20641d828ec8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '523ed589872a45a09b6a7675a7fb0b75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:19.455498 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9da305-483e-4489-9343-20641d828ec8 service cinder] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:19.455771 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9da305-483e-4489-9343-20641d828ec8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:19.456037 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9da305-483e-4489-9343-20641d828ec8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:19.465529 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9da305-483e-4489-9343-20641d828ec8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:19.466648 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9da305-483e-4489-9343-20641d828ec8 service cinder] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:19.476394 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1109/2243] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:32:19 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:32:29.311601 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=65,keystone:INSERT=4 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:32:29.471874 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=10 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:32:33.421753 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1725c59-9fee-4bd2-a4bb-cfdede1b19cf None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:33.422247 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1725c59-9fee-4bd2-a4bb-cfdede1b19cf None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:33.422327 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1725c59-9fee-4bd2-a4bb-cfdede1b19cf None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:33.424290 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1135/2244] 10.222.0.102 () {54 vars in 926 bytes} [Thu Sep 1 17:32:33 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:32:33.435724 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621d1394-d15b-4e9b-9f4b-f7713b4dd531 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:33.436172 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621d1394-d15b-4e9b-9f4b-f7713b4dd531 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:33.436277 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621d1394-d15b-4e9b-9f4b-f7713b4dd531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:33.455643 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-621d1394-d15b-4e9b-9f4b-f7713b4dd531 None None] MFA Rules not processed for user `134ef6039cef468bae0e3f3be79b8073`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:32:33.456891 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-621d1394-d15b-4e9b-9f4b-f7713b4dd531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:33.538136 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1110/2245] 10.222.0.102 () {58 vars in 1023 bytes} [Thu Sep 1 17:32:33 2022] POST /identity/v3/auth/tokens => generated 3318 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:32:33.551740 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1725c59-9fee-4bd2-a4bb-cfdede1b19cf None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:33.554239 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-a1725c59-9fee-4bd2-a4bb-cfdede1b19cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:33.571830 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1725c59-9fee-4bd2-a4bb-cfdede1b19cf None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:33.577078 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3bb52a-61d0-4b58-ac30-bf2ad6566409 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '134ef6039cef468bae0e3f3be79b8073', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:33.577896 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3bb52a-61d0-4b58-ac30-bf2ad6566409 service placement] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:33.578177 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3bb52a-61d0-4b58-ac30-bf2ad6566409 service placement] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:33.578454 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3bb52a-61d0-4b58-ac30-bf2ad6566409 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:33.586939 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3bb52a-61d0-4b58-ac30-bf2ad6566409 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:33.588978 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3bb52a-61d0-4b58-ac30-bf2ad6566409 service placement] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:33.599598 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1136/2246] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:32:33 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:32:34.015260 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621d1394-d15b-4e9b-9f4b-f7713b4dd531 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:34.102288 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621d1394-d15b-4e9b-9f4b-f7713b4dd531 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:34.106828 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64d8ef8-2842-474a-95a1-592a776a3ee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:34.107528 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64d8ef8-2842-474a-95a1-592a776a3ee7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:34.107879 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64d8ef8-2842-474a-95a1-592a776a3ee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:34.108221 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64d8ef8-2842-474a-95a1-592a776a3ee7 admin admin] PATH_INFO: `/v3/users/31cab3856ca041db82d1caf5aeeaadd0` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:34.109461 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64d8ef8-2842-474a-95a1-592a776a3ee7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=31cab3856ca041db82d1caf5aeeaadd0)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:34.111478 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64d8ef8-2842-474a-95a1-592a776a3ee7 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:34.165509 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-b64d8ef8-2842-474a-95a1-592a776a3ee7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31cab3856ca041db82d1caf5aeeaadd0'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:34.166251 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-b64d8ef8-2842-474a-95a1-592a776a3ee7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31cab3856ca041db82d1caf5aeeaadd0'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:34.181522 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-b64d8ef8-2842-474a-95a1-592a776a3ee7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31cab3856ca041db82d1caf5aeeaadd0'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:34.189100 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-b64d8ef8-2842-474a-95a1-592a776a3ee7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31cab3856ca041db82d1caf5aeeaadd0'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:34.189730 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-b64d8ef8-2842-474a-95a1-592a776a3ee7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31cab3856ca041db82d1caf5aeeaadd0'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:34.206956 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1111/2247] 10.222.0.102 () {58 vars in 1229 bytes} [Thu Sep 1 17:32:34 2022] DELETE /identity/v3/users/31cab3856ca041db82d1caf5aeeaadd0 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:32:34.211368 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6124523c-8c39-4957-a9ed-e029bb00bfa4 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:34.211658 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6124523c-8c39-4957-a9ed-e029bb00bfa4 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:34.211936 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6124523c-8c39-4957-a9ed-e029bb00bfa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:34.233294 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-6124523c-8c39-4957-a9ed-e029bb00bfa4 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:32:34.234655 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-6124523c-8c39-4957-a9ed-e029bb00bfa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:34.294122 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64d8ef8-2842-474a-95a1-592a776a3ee7 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:34.294122 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-b64d8ef8-2842-474a-95a1-592a776a3ee7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:34.302119 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1137/2248] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:32:34 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:32:34.313100 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6124523c-8c39-4957-a9ed-e029bb00bfa4 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:34.316128 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-6124523c-8c39-4957-a9ed-e029bb00bfa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:34.323628 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:32:34.380781 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6124523c-8c39-4957-a9ed-e029bb00bfa4 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:34.384802 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85122c4-e6a7-4e11-8570-f8cba6e5575a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:34.386180 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85122c4-e6a7-4e11-8570-f8cba6e5575a service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:34.386465 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85122c4-e6a7-4e11-8570-f8cba6e5575a service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:34.387016 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85122c4-e6a7-4e11-8570-f8cba6e5575a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:34.388714 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64d8ef8-2842-474a-95a1-592a776a3ee7 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:34.394084 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83911193-b6ad-4d33-b3c5-a93eba461ff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:34.395315 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83911193-b6ad-4d33-b3c5-a93eba461ff6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:34.395814 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83911193-b6ad-4d33-b3c5-a93eba461ff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:34.396991 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83911193-b6ad-4d33-b3c5-a93eba461ff6 admin admin] PATH_INFO: `/v3/users/d7bd83f1f7e24df2890ab3fc1aabb60d` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:34.399438 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83911193-b6ad-4d33-b3c5-a93eba461ff6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7bd83f1f7e24df2890ab3fc1aabb60d)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:34.401629 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85122c4-e6a7-4e11-8570-f8cba6e5575a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:34.402138 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83911193-b6ad-4d33-b3c5-a93eba461ff6 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:34.403750 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85122c4-e6a7-4e11-8570-f8cba6e5575a service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:34.417004 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1138/2249] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:32:34 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:32:34.464357 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-83911193-b6ad-4d33-b3c5-a93eba461ff6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7bd83f1f7e24df2890ab3fc1aabb60d'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:34.465897 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-83911193-b6ad-4d33-b3c5-a93eba461ff6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7bd83f1f7e24df2890ab3fc1aabb60d'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:34.482022 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-83911193-b6ad-4d33-b3c5-a93eba461ff6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7bd83f1f7e24df2890ab3fc1aabb60d'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:34.503379 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-83911193-b6ad-4d33-b3c5-a93eba461ff6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7bd83f1f7e24df2890ab3fc1aabb60d'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:34.505132 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-83911193-b6ad-4d33-b3c5-a93eba461ff6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7bd83f1f7e24df2890ab3fc1aabb60d'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:34.539921 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1112/2250] 10.222.0.102 () {58 vars in 1229 bytes} [Thu Sep 1 17:32:34 2022] DELETE /identity/v3/users/d7bd83f1f7e24df2890ab3fc1aabb60d => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:32:34.940594 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85122c4-e6a7-4e11-8570-f8cba6e5575a service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:34.943101 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-c85122c4-e6a7-4e11-8570-f8cba6e5575a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:34.955847 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83911193-b6ad-4d33-b3c5-a93eba461ff6 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:34.957254 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-83911193-b6ad-4d33-b3c5-a93eba461ff6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:34.965407 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85122c4-e6a7-4e11-8570-f8cba6e5575a service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:35.047471 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d59b360-6232-4b1c-b7fc-342a7a442d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:35.048216 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d59b360-6232-4b1c-b7fc-342a7a442d2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:35.048498 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d59b360-6232-4b1c-b7fc-342a7a442d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:35.048781 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d59b360-6232-4b1c-b7fc-342a7a442d2c admin admin] PATH_INFO: `/v3/projects/e5d2bf10be5a47bba216ed775f0837e8` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:35.050246 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d59b360-6232-4b1c-b7fc-342a7a442d2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5d2bf10be5a47bba216ed775f0837e8)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:35.052490 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d59b360-6232-4b1c-b7fc-342a7a442d2c admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:35.064189 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83911193-b6ad-4d33-b3c5-a93eba461ff6 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:35.068766 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d4f734-5760-47e0-a11b-70653c989d47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:35.069428 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d4f734-5760-47e0-a11b-70653c989d47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:35.069709 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d4f734-5760-47e0-a11b-70653c989d47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:35.069979 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d4f734-5760-47e0-a11b-70653c989d47 admin admin] PATH_INFO: `/v3/projects/0ba49950cc214014a48b1829c862a8d4` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:35.071114 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d4f734-5760-47e0-a11b-70653c989d47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ba49950cc214014a48b1829c862a8d4)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:35.073048 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d4f734-5760-47e0-a11b-70653c989d47 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:35.103483 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:32:35.123262 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-1d59b360-6232-4b1c-b7fc-342a7a442d2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5d2bf10be5a47bba216ed775f0837e8', 'request_id': 'req-1d59b360-6232-4b1c-b7fc-342a7a442d2c'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:35.129352 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:32:35.140195 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-1d59b360-6232-4b1c-b7fc-342a7a442d2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5d2bf10be5a47bba216ed775f0837e8', 'request_id': 'req-1d59b360-6232-4b1c-b7fc-342a7a442d2c'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:35.146264 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-77d4f734-5760-47e0-a11b-70653c989d47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ba49950cc214014a48b1829c862a8d4', 'request_id': 'req-77d4f734-5760-47e0-a11b-70653c989d47'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:35.148918 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-1d59b360-6232-4b1c-b7fc-342a7a442d2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5d2bf10be5a47bba216ed775f0837e8', 'request_id': 'req-1d59b360-6232-4b1c-b7fc-342a7a442d2c'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:35.156283 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-77d4f734-5760-47e0-a11b-70653c989d47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ba49950cc214014a48b1829c862a8d4', 'request_id': 'req-77d4f734-5760-47e0-a11b-70653c989d47'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:35.164690 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-1d59b360-6232-4b1c-b7fc-342a7a442d2c admin admin] The token cache is being invalidate because project e5d2bf10be5a47bba216ed775f0837e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:32:35.165691 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-1d59b360-6232-4b1c-b7fc-342a7a442d2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:35.167340 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-1d59b360-6232-4b1c-b7fc-342a7a442d2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:35.168577 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-77d4f734-5760-47e0-a11b-70653c989d47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ba49950cc214014a48b1829c862a8d4', 'request_id': 'req-77d4f734-5760-47e0-a11b-70653c989d47'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:35.169609 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1139/2251] 10.222.0.102 () {58 vars in 1238 bytes} [Thu Sep 1 17:32:34 2022] DELETE /identity/v3/projects/e5d2bf10be5a47bba216ed775f0837e8 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:32:35.188079 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-77d4f734-5760-47e0-a11b-70653c989d47 admin admin] The token cache is being invalidate because project 0ba49950cc214014a48b1829c862a8d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:32:35.188392 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-77d4f734-5760-47e0-a11b-70653c989d47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:35.189469 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-77d4f734-5760-47e0-a11b-70653c989d47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:35.191694 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1113/2252] 10.222.0.102 () {58 vars in 1238 bytes} [Thu Sep 1 17:32:34 2022] DELETE /identity/v3/projects/0ba49950cc214014a48b1829c862a8d4 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:32:35.195549 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d59b360-6232-4b1c-b7fc-342a7a442d2c admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:35.197448 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-1d59b360-6232-4b1c-b7fc-342a7a442d2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:35.296506 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d59b360-6232-4b1c-b7fc-342a7a442d2c admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:35.362830 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756d2dd5-d2fd-4f5c-8cc2-3b7aca9e48b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:35.363497 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756d2dd5-d2fd-4f5c-8cc2-3b7aca9e48b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:35.363775 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756d2dd5-d2fd-4f5c-8cc2-3b7aca9e48b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:35.364091 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756d2dd5-d2fd-4f5c-8cc2-3b7aca9e48b9 admin admin] PATH_INFO: `/v3/users/4b04c3d415514ec38d4b28d19d8504e2` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:35.365216 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756d2dd5-d2fd-4f5c-8cc2-3b7aca9e48b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b04c3d415514ec38d4b28d19d8504e2)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:35.367192 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756d2dd5-d2fd-4f5c-8cc2-3b7aca9e48b9 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:35.415070 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-756d2dd5-d2fd-4f5c-8cc2-3b7aca9e48b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b04c3d415514ec38d4b28d19d8504e2'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:35.426658 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-756d2dd5-d2fd-4f5c-8cc2-3b7aca9e48b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b04c3d415514ec38d4b28d19d8504e2'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:35.430215 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:32:35.435212 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-756d2dd5-d2fd-4f5c-8cc2-3b7aca9e48b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b04c3d415514ec38d4b28d19d8504e2'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:35.435967 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-756d2dd5-d2fd-4f5c-8cc2-3b7aca9e48b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b04c3d415514ec38d4b28d19d8504e2'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:35.436886 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-756d2dd5-d2fd-4f5c-8cc2-3b7aca9e48b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b04c3d415514ec38d4b28d19d8504e2'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:35.450648 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1140/2253] 10.222.0.102 () {58 vars in 1229 bytes} [Thu Sep 1 17:32:35 2022] DELETE /identity/v3/users/4b04c3d415514ec38d4b28d19d8504e2 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:32:35.624166 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d4f734-5760-47e0-a11b-70653c989d47 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:35.625736 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-77d4f734-5760-47e0-a11b-70653c989d47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:35.639979 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d4f734-5760-47e0-a11b-70653c989d47 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:35.714051 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779a9a42-c207-49f3-b853-e274c023a839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:35.714692 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779a9a42-c207-49f3-b853-e274c023a839 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:35.714966 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779a9a42-c207-49f3-b853-e274c023a839 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:35.715230 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779a9a42-c207-49f3-b853-e274c023a839 admin admin] PATH_INFO: `/v3/projects/57bbcfd7a34d41d3b27d332c5adede46` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:35.716369 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779a9a42-c207-49f3-b853-e274c023a839 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57bbcfd7a34d41d3b27d332c5adede46)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:35.718790 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779a9a42-c207-49f3-b853-e274c023a839 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:35.770416 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-779a9a42-c207-49f3-b853-e274c023a839 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57bbcfd7a34d41d3b27d332c5adede46', 'request_id': 'req-779a9a42-c207-49f3-b853-e274c023a839'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:35.777770 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-779a9a42-c207-49f3-b853-e274c023a839 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57bbcfd7a34d41d3b27d332c5adede46', 'request_id': 'req-779a9a42-c207-49f3-b853-e274c023a839'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:35.785796 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-779a9a42-c207-49f3-b853-e274c023a839 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57bbcfd7a34d41d3b27d332c5adede46', 'request_id': 'req-779a9a42-c207-49f3-b853-e274c023a839'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:35.798618 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-779a9a42-c207-49f3-b853-e274c023a839 admin admin] The token cache is being invalidate because project 57bbcfd7a34d41d3b27d332c5adede46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:32:35.798959 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-779a9a42-c207-49f3-b853-e274c023a839 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:35.799624 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-779a9a42-c207-49f3-b853-e274c023a839 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:35.801441 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1114/2254] 10.222.0.102 () {58 vars in 1238 bytes} [Thu Sep 1 17:32:35 2022] DELETE /identity/v3/projects/57bbcfd7a34d41d3b27d332c5adede46 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:32:35.806862 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e6dc36-90b6-497a-beaa-2ce869adf979 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:35.807767 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e6dc36-90b6-497a-beaa-2ce869adf979 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:35.808117 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e6dc36-90b6-497a-beaa-2ce869adf979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:35.841568 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-c2e6dc36-90b6-497a-beaa-2ce869adf979 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:32:35.843228 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-c2e6dc36-90b6-497a-beaa-2ce869adf979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:35.917382 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1141/2255] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:32:35 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:32:35.926104 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fe7533-a77a-4faa-8b0f-3cd4e9f7d6a6 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:35.926333 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fe7533-a77a-4faa-8b0f-3cd4e9f7d6a6 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:35.926637 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fe7533-a77a-4faa-8b0f-3cd4e9f7d6a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:35.948916 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-47fe7533-a77a-4faa-8b0f-3cd4e9f7d6a6 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:32:35.950146 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-47fe7533-a77a-4faa-8b0f-3cd4e9f7d6a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:35.962571 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e6dc36-90b6-497a-beaa-2ce869adf979 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:35.967954 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1115/2256] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:32:35 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:32:35.973225 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e6dc36-90b6-497a-beaa-2ce869adf979 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:35.978574 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3068d0e5-f8a8-4b32-ab59-59eab1f23727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:35.979326 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3068d0e5-f8a8-4b32-ab59-59eab1f23727 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:35.979689 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3068d0e5-f8a8-4b32-ab59-59eab1f23727 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:35.980028 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3068d0e5-f8a8-4b32-ab59-59eab1f23727 admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:35.980750 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3068d0e5-f8a8-4b32-ab59-59eab1f23727 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:32:35.981118 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3068d0e5-f8a8-4b32-ab59-59eab1f23727 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:35.983079 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3068d0e5-f8a8-4b32-ab59-59eab1f23727 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:35.997433 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1142/2257] 10.222.0.102 () {58 vars in 1158 bytes} [Thu Sep 1 17:32:35 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:32:36.001386 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fe7533-a77a-4faa-8b0f-3cd4e9f7d6a6 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:36.011725 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fe7533-a77a-4faa-8b0f-3cd4e9f7d6a6 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:36.017675 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f095c2-bd58-4a42-9778-43d0ce86931d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:36.018508 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f095c2-bd58-4a42-9778-43d0ce86931d admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:36.018972 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f095c2-bd58-4a42-9778-43d0ce86931d admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:36.019612 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f095c2-bd58-4a42-9778-43d0ce86931d admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:36.020426 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f095c2-bd58-4a42-9778-43d0ce86931d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:36.022566 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f095c2-bd58-4a42-9778-43d0ce86931d admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:36.052427 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1116/2258] 10.222.0.102 () {60 vars in 1158 bytes} [Thu Sep 1 17:32:36 2022] POST /identity/v3/projects => generated 378 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:32:36.056873 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3068d0e5-f8a8-4b32-ab59-59eab1f23727 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:36.068369 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3068d0e5-f8a8-4b32-ab59-59eab1f23727 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:36.134136 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f292c220-7f0c-4f12-a1f2-98795e08665a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:36.135036 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f292c220-7f0c-4f12-a1f2-98795e08665a admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:36.135360 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f292c220-7f0c-4f12-a1f2-98795e08665a admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:36.135630 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f292c220-7f0c-4f12-a1f2-98795e08665a admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:36.136279 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f292c220-7f0c-4f12-a1f2-98795e08665a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:36.138278 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f292c220-7f0c-4f12-a1f2-98795e08665a admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:36.164754 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1143/2259] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:32:36 2022] POST /identity/v3/users => generated 290 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:32:36.166137 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f095c2-bd58-4a42-9778-43d0ce86931d admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:36.177705 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f095c2-bd58-4a42-9778-43d0ce86931d admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:36.182348 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c01bd8-43e0-4121-9a5e-389c074481b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:36.182955 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c01bd8-43e0-4121-9a5e-389c074481b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:36.183237 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c01bd8-43e0-4121-9a5e-389c074481b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:36.183510 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c01bd8-43e0-4121-9a5e-389c074481b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:36.184149 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c01bd8-43e0-4121-9a5e-389c074481b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:36.185991 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c01bd8-43e0-4121-9a5e-389c074481b0 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:36.201002 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1117/2260] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:32:36 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:32:36.202215 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f292c220-7f0c-4f12-a1f2-98795e08665a admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:36.212926 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f292c220-7f0c-4f12-a1f2-98795e08665a admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:36.219534 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004c29fd-2668-4af6-a9ff-e07cb59c88c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:36.220440 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004c29fd-2668-4af6-a9ff-e07cb59c88c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:36.221661 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004c29fd-2668-4af6-a9ff-e07cb59c88c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:36.221661 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004c29fd-2668-4af6-a9ff-e07cb59c88c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:36.221836 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004c29fd-2668-4af6-a9ff-e07cb59c88c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:36.223496 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004c29fd-2668-4af6-a9ff-e07cb59c88c1 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:36.237147 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1144/2261] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:32:36 2022] GET /identity/v3/roles => generated 1427 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:32:36.248299 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c01bd8-43e0-4121-9a5e-389c074481b0 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:36.258868 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c01bd8-43e0-4121-9a5e-389c074481b0 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:36.269383 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e873ef-a156-4bd1-8f5c-33bb9028afaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:36.271173 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e873ef-a156-4bd1-8f5c-33bb9028afaf admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:36.271582 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e873ef-a156-4bd1-8f5c-33bb9028afaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:36.272069 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e873ef-a156-4bd1-8f5c-33bb9028afaf admin admin] PATH_INFO: `/v3/projects/0a69f0aa0eff449f9802515e7a5960fd/users/9d638d98e53e4d81a0b18218e3acdc25/roles/b75cb360925e4ba5a65f14965c103ffd` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:36.292965 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:32:36.297816 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e873ef-a156-4bd1-8f5c-33bb9028afaf admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a69f0aa0eff449f9802515e7a5960fd, user_id=9d638d98e53e4d81a0b18218e3acdc25, role_id=b75cb360925e4ba5a65f14965c103ffd)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:36.301505 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e873ef-a156-4bd1-8f5c-33bb9028afaf admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:36.316776 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1118/2262] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:32:36 2022] PUT /identity/v3/projects/0a69f0aa0eff449f9802515e7a5960fd/users/9d638d98e53e4d81a0b18218e3acdc25/roles/b75cb360925e4ba5a65f14965c103ffd => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:32:36.327259 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7475d5ec-871d-465d-8a91-a3d48becd5cf None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:36.327629 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7475d5ec-871d-465d-8a91-a3d48becd5cf None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:36.327995 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7475d5ec-871d-465d-8a91-a3d48becd5cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:36.358325 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-7475d5ec-871d-465d-8a91-a3d48becd5cf None None] MFA Rules not processed for user `9d638d98e53e4d81a0b18218e3acdc25`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:32:36.359597 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-7475d5ec-871d-465d-8a91-a3d48becd5cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:36.412077 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:32:36.441922 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1145/2263] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:32:36 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:32:36.446320 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e873ef-a156-4bd1-8f5c-33bb9028afaf admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:36.517686 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e873ef-a156-4bd1-8f5c-33bb9028afaf admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:36.524352 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ec5991-9287-42f7-8414-3a9009d1fd32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:36.524959 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ec5991-9287-42f7-8414-3a9009d1fd32 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:36.525299 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ec5991-9287-42f7-8414-3a9009d1fd32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:36.525533 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ec5991-9287-42f7-8414-3a9009d1fd32 admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:36.526191 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ec5991-9287-42f7-8414-3a9009d1fd32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:36.528108 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ec5991-9287-42f7-8414-3a9009d1fd32 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:36.558787 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1119/2264] 10.222.0.102 () {60 vars in 1158 bytes} [Thu Sep 1 17:32:36 2022] POST /identity/v3/projects => generated 378 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:32:36.562429 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7475d5ec-871d-465d-8a91-a3d48becd5cf None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:36.574026 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7475d5ec-871d-465d-8a91-a3d48becd5cf None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:36.650245 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebed3d5-dc8b-4a75-8c72-5aedb8455c9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:36.651549 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebed3d5-dc8b-4a75-8c72-5aedb8455c9f admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:36.651922 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebed3d5-dc8b-4a75-8c72-5aedb8455c9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:36.652247 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebed3d5-dc8b-4a75-8c72-5aedb8455c9f admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:36.652959 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebed3d5-dc8b-4a75-8c72-5aedb8455c9f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:36.655451 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebed3d5-dc8b-4a75-8c72-5aedb8455c9f admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:36.679958 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1146/2265] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:32:36 2022] POST /identity/v3/users => generated 290 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:32:36.684650 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ec5991-9287-42f7-8414-3a9009d1fd32 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:36.698287 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ec5991-9287-42f7-8414-3a9009d1fd32 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:36.703584 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6288916-e977-4d5b-af61-bc8fbe30370a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:36.708971 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6288916-e977-4d5b-af61-bc8fbe30370a admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:36.708971 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6288916-e977-4d5b-af61-bc8fbe30370a admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:36.708971 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6288916-e977-4d5b-af61-bc8fbe30370a admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:36.708971 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6288916-e977-4d5b-af61-bc8fbe30370a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:36.708971 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6288916-e977-4d5b-af61-bc8fbe30370a admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:36.725401 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1120/2266] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:32:36 2022] GET /identity/v3/roles => generated 1427 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:32:36.728311 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebed3d5-dc8b-4a75-8c72-5aedb8455c9f admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:36.740097 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebed3d5-dc8b-4a75-8c72-5aedb8455c9f admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:36.745211 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6616b0fe-1290-48c2-8616-397f53eb898b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:36.749621 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6616b0fe-1290-48c2-8616-397f53eb898b admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:36.749957 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6616b0fe-1290-48c2-8616-397f53eb898b admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:36.750241 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6616b0fe-1290-48c2-8616-397f53eb898b admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:36.750965 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6616b0fe-1290-48c2-8616-397f53eb898b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:36.752606 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6616b0fe-1290-48c2-8616-397f53eb898b admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:36.770158 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1147/2267] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:32:36 2022] GET /identity/v3/roles => generated 1427 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:32:36.771416 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6288916-e977-4d5b-af61-bc8fbe30370a admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:36.782487 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6288916-e977-4d5b-af61-bc8fbe30370a admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:36.787308 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f43752a-8907-4b8b-a313-5ff352962f8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:36.787949 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f43752a-8907-4b8b-a313-5ff352962f8f admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:36.788226 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f43752a-8907-4b8b-a313-5ff352962f8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:36.788498 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f43752a-8907-4b8b-a313-5ff352962f8f admin admin] PATH_INFO: `/v3/projects/53bbf3bbe4c04adaa9d540ecdf769b32/users/2d1808a56ada4a7ab22cf1c39aa154f9/roles/b75cb360925e4ba5a65f14965c103ffd` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:36.803345 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f43752a-8907-4b8b-a313-5ff352962f8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=53bbf3bbe4c04adaa9d540ecdf769b32, user_id=2d1808a56ada4a7ab22cf1c39aa154f9, role_id=b75cb360925e4ba5a65f14965c103ffd)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:36.805998 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f43752a-8907-4b8b-a313-5ff352962f8f admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:36.819069 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1121/2268] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:32:36 2022] PUT /identity/v3/projects/53bbf3bbe4c04adaa9d540ecdf769b32/users/2d1808a56ada4a7ab22cf1c39aa154f9/roles/b75cb360925e4ba5a65f14965c103ffd => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:32:36.829775 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a36daea-9ea4-4aa5-9e00-0c7e306b102b None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:36.830056 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a36daea-9ea4-4aa5-9e00-0c7e306b102b None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:36.830326 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a36daea-9ea4-4aa5-9e00-0c7e306b102b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:36.865480 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-4a36daea-9ea4-4aa5-9e00-0c7e306b102b None None] MFA Rules not processed for user `2d1808a56ada4a7ab22cf1c39aa154f9`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:32:36.866839 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-4a36daea-9ea4-4aa5-9e00-0c7e306b102b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:36.943387 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1148/2269] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:32:36 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:32:36.945725 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f43752a-8907-4b8b-a313-5ff352962f8f admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:37.011997 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f43752a-8907-4b8b-a313-5ff352962f8f admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:37.018246 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc89865-de7e-4f00-80d1-0679f3fc2e23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:37.018892 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc89865-de7e-4f00-80d1-0679f3fc2e23 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:37.019147 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc89865-de7e-4f00-80d1-0679f3fc2e23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:37.019420 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc89865-de7e-4f00-80d1-0679f3fc2e23 admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:37.026834 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc89865-de7e-4f00-80d1-0679f3fc2e23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:37.032256 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc89865-de7e-4f00-80d1-0679f3fc2e23 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:37.091857 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1122/2270] 10.222.0.102 () {60 vars in 1158 bytes} [Thu Sep 1 17:32:36 2022] POST /identity/v3/projects => generated 376 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:32:37.091982 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a36daea-9ea4-4aa5-9e00-0c7e306b102b None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:37.101109 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a36daea-9ea4-4aa5-9e00-0c7e306b102b None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:37.147515 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:32:37.158929 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a874748-40ae-46c1-b4fb-63d88b34e77f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:37.159550 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a874748-40ae-46c1-b4fb-63d88b34e77f admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:37.159835 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a874748-40ae-46c1-b4fb-63d88b34e77f admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:37.160113 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a874748-40ae-46c1-b4fb-63d88b34e77f admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:37.160769 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a874748-40ae-46c1-b4fb-63d88b34e77f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:37.162787 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a874748-40ae-46c1-b4fb-63d88b34e77f admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:37.183052 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1149/2271] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:32:37 2022] POST /identity/v3/users => generated 295 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:32:37.187152 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc89865-de7e-4f00-80d1-0679f3fc2e23 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:37.197283 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc89865-de7e-4f00-80d1-0679f3fc2e23 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:37.202305 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b432474f-88f9-42a5-8175-7552b581f2eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:37.202888 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b432474f-88f9-42a5-8175-7552b581f2eb admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:37.203164 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b432474f-88f9-42a5-8175-7552b581f2eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:37.203432 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b432474f-88f9-42a5-8175-7552b581f2eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:37.204061 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b432474f-88f9-42a5-8175-7552b581f2eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:37.205660 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b432474f-88f9-42a5-8175-7552b581f2eb admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:37.216766 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1123/2272] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:32:37 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:32:37.221136 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a874748-40ae-46c1-b4fb-63d88b34e77f admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:37.231233 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a874748-40ae-46c1-b4fb-63d88b34e77f admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:37.235883 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670ed274-ce36-4995-aa26-2c3ec3f1e16f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:37.236544 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670ed274-ce36-4995-aa26-2c3ec3f1e16f admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:37.236851 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670ed274-ce36-4995-aa26-2c3ec3f1e16f admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:37.237154 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670ed274-ce36-4995-aa26-2c3ec3f1e16f admin admin] PATH_INFO: `/v3/projects/60d4b903a6ef4febbfbf650bc85be686/users/33bb714ea00947138bb5a9c29c03f7c6/roles/36577a6f46d642d4b81805ede9872490` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:37.254702 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670ed274-ce36-4995-aa26-2c3ec3f1e16f admin admin] RBAC: Authorizing `identity:create_grant(project_id=60d4b903a6ef4febbfbf650bc85be686, user_id=33bb714ea00947138bb5a9c29c03f7c6, role_id=36577a6f46d642d4b81805ede9872490)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:37.257487 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670ed274-ce36-4995-aa26-2c3ec3f1e16f admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:37.271180 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1150/2273] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:32:37 2022] PUT /identity/v3/projects/60d4b903a6ef4febbfbf650bc85be686/users/33bb714ea00947138bb5a9c29c03f7c6/roles/36577a6f46d642d4b81805ede9872490 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:32:37.275105 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b432474f-88f9-42a5-8175-7552b581f2eb admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:37.285521 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b432474f-88f9-42a5-8175-7552b581f2eb admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:37.296764 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:32:37.345729 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a811f7ac-0ed2-4bec-9131-453947820651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:37.346334 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a811f7ac-0ed2-4bec-9131-453947820651 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:37.346619 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a811f7ac-0ed2-4bec-9131-453947820651 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:37.346886 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a811f7ac-0ed2-4bec-9131-453947820651 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:37.347518 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a811f7ac-0ed2-4bec-9131-453947820651 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:37.349101 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a811f7ac-0ed2-4bec-9131-453947820651 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:37.358904 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1124/2274] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:32:37 2022] GET /identity/v3/roles => generated 1427 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:32:37.363069 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670ed274-ce36-4995-aa26-2c3ec3f1e16f admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:37.373668 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670ed274-ce36-4995-aa26-2c3ec3f1e16f admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:37.378287 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5f085c-18d4-4690-b86b-9a540f398f4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:37.378926 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5f085c-18d4-4690-b86b-9a540f398f4f admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:37.379206 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5f085c-18d4-4690-b86b-9a540f398f4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:37.379475 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5f085c-18d4-4690-b86b-9a540f398f4f admin admin] PATH_INFO: `/v3/projects/60d4b903a6ef4febbfbf650bc85be686/users/33bb714ea00947138bb5a9c29c03f7c6/roles/36577a6f46d642d4b81805ede9872490` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:37.381428 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5f085c-18d4-4690-b86b-9a540f398f4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=60d4b903a6ef4febbfbf650bc85be686, user_id=33bb714ea00947138bb5a9c29c03f7c6, role_id=36577a6f46d642d4b81805ede9872490)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:37.384055 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5f085c-18d4-4690-b86b-9a540f398f4f admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:37.394864 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1151/2275] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:32:37 2022] PUT /identity/v3/projects/60d4b903a6ef4febbfbf650bc85be686/users/33bb714ea00947138bb5a9c29c03f7c6/roles/36577a6f46d642d4b81805ede9872490 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:32:37.405863 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134f0adc-9111-4eed-89bd-4a559ad1977f None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:37.406230 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134f0adc-9111-4eed-89bd-4a559ad1977f None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:37.406579 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134f0adc-9111-4eed-89bd-4a559ad1977f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:37.436590 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-134f0adc-9111-4eed-89bd-4a559ad1977f None None] MFA Rules not processed for user `33bb714ea00947138bb5a9c29c03f7c6`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:32:37.437871 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-134f0adc-9111-4eed-89bd-4a559ad1977f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:37.511284 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1125/2276] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:32:37 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:32:37.523864 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5f085c-18d4-4690-b86b-9a540f398f4f admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:37.527182 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-1f5f085c-18d4-4690-b86b-9a540f398f4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:37.601632 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5f085c-18d4-4690-b86b-9a540f398f4f admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:37.605443 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5c60e6-33bb-4e81-bf3c-f1f771c37d25 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '523ed589872a45a09b6a7675a7fb0b75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:37.606050 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5c60e6-33bb-4e81-bf3c-f1f771c37d25 service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:37.606330 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5c60e6-33bb-4e81-bf3c-f1f771c37d25 service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:37.606603 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5c60e6-33bb-4e81-bf3c-f1f771c37d25 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:37.662033 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5c60e6-33bb-4e81-bf3c-f1f771c37d25 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:37.663468 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5c60e6-33bb-4e81-bf3c-f1f771c37d25 service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:37.672431 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1152/2277] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:32:37 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:32:40.289152 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134f0adc-9111-4eed-89bd-4a559ad1977f None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:40.299162 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134f0adc-9111-4eed-89bd-4a559ad1977f None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:40.302707 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64699d49-7861-4883-990c-cf81eec56d06 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '523ed589872a45a09b6a7675a7fb0b75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:40.303298 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64699d49-7861-4883-990c-cf81eec56d06 service cinder] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:40.303580 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64699d49-7861-4883-990c-cf81eec56d06 service cinder] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:40.303879 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64699d49-7861-4883-990c-cf81eec56d06 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:40.361344 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64699d49-7861-4883-990c-cf81eec56d06 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:40.362777 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64699d49-7861-4883-990c-cf81eec56d06 service cinder] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:40.371601 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1126/2278] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:32:40 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:32:41.042225 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5c60e6-33bb-4e81-bf3c-f1f771c37d25 service cinder] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:41.050826 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5c60e6-33bb-4e81-bf3c-f1f771c37d25 service cinder] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:41.054494 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c65619-43de-4df0-9a88-d55dbf970746 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '523ed589872a45a09b6a7675a7fb0b75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:41.055099 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c65619-43de-4df0-9a88-d55dbf970746 service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:41.055356 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c65619-43de-4df0-9a88-d55dbf970746 service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:41.055624 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c65619-43de-4df0-9a88-d55dbf970746 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:41.064823 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c65619-43de-4df0-9a88-d55dbf970746 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:41.066266 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c65619-43de-4df0-9a88-d55dbf970746 service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:41.076021 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1153/2279] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:32:41 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:32:46.074441 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb82b6f-feaa-4dea-961d-44972bc3c11f None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:46.074954 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb82b6f-feaa-4dea-961d-44972bc3c11f None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:46.075339 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb82b6f-feaa-4dea-961d-44972bc3c11f None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:46.077166 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1127/2280] 10.222.0.102 () {54 vars in 926 bytes} [Thu Sep 1 17:32:46 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:32:46.101661 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31649b4-8ee3-4183-b275-a7ca5122b076 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:46.102107 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31649b4-8ee3-4183-b275-a7ca5122b076 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:46.103203 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31649b4-8ee3-4183-b275-a7ca5122b076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:46.137099 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-e31649b4-8ee3-4183-b275-a7ca5122b076 None None] MFA Rules not processed for user `134ef6039cef468bae0e3f3be79b8073`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:32:46.137962 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb82b6f-feaa-4dea-961d-44972bc3c11f None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:46.138360 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-e31649b4-8ee3-4183-b275-a7ca5122b076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:46.139949 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-2cb82b6f-feaa-4dea-961d-44972bc3c11f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:46.184784 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:32:46.189465 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:32:46.212295 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1154/2281] 10.222.0.102 () {58 vars in 1023 bytes} [Thu Sep 1 17:32:46 2022] POST /identity/v3/auth/tokens => generated 3318 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:32:46.213110 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31649b4-8ee3-4183-b275-a7ca5122b076 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:46.220377 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb82b6f-feaa-4dea-961d-44972bc3c11f None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:46.226489 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a199e5c0-0fcf-4635-8c71-37672d3f3084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:46.227019 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a199e5c0-0fcf-4635-8c71-37672d3f3084 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:46.227341 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a199e5c0-0fcf-4635-8c71-37672d3f3084 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:46.227719 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a199e5c0-0fcf-4635-8c71-37672d3f3084 admin admin] PATH_INFO: `/v3/users/19a53d1e1a84408ea1dc2aba24bd8299` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:46.229628 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a199e5c0-0fcf-4635-8c71-37672d3f3084 admin admin] RBAC: Authorizing `identity:delete_user(user_id=19a53d1e1a84408ea1dc2aba24bd8299)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:46.231947 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31649b4-8ee3-4183-b275-a7ca5122b076 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:46.232325 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a199e5c0-0fcf-4635-8c71-37672d3f3084 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:46.238229 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ff72bd-0b65-401f-b0ce-534285d30df8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:46.239277 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ff72bd-0b65-401f-b0ce-534285d30df8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:46.239674 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ff72bd-0b65-401f-b0ce-534285d30df8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:46.240008 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ff72bd-0b65-401f-b0ce-534285d30df8 admin admin] PATH_INFO: `/v3/users/9d638d98e53e4d81a0b18218e3acdc25` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:46.242068 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ff72bd-0b65-401f-b0ce-534285d30df8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d638d98e53e4d81a0b18218e3acdc25)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:46.244008 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ff72bd-0b65-401f-b0ce-534285d30df8 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:46.295849 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-a199e5c0-0fcf-4635-8c71-37672d3f3084 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19a53d1e1a84408ea1dc2aba24bd8299'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:46.296779 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-a199e5c0-0fcf-4635-8c71-37672d3f3084 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19a53d1e1a84408ea1dc2aba24bd8299'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:46.307459 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-d4ff72bd-0b65-401f-b0ce-534285d30df8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d638d98e53e4d81a0b18218e3acdc25'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:46.312958 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-a199e5c0-0fcf-4635-8c71-37672d3f3084 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19a53d1e1a84408ea1dc2aba24bd8299'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:46.321200 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-a199e5c0-0fcf-4635-8c71-37672d3f3084 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19a53d1e1a84408ea1dc2aba24bd8299'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:46.322011 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-a199e5c0-0fcf-4635-8c71-37672d3f3084 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19a53d1e1a84408ea1dc2aba24bd8299'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:46.326419 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-d4ff72bd-0b65-401f-b0ce-534285d30df8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d638d98e53e4d81a0b18218e3acdc25'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:46.335672 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1128/2282] 10.222.0.102 () {58 vars in 1229 bytes} [Thu Sep 1 17:32:46 2022] DELETE /identity/v3/users/19a53d1e1a84408ea1dc2aba24bd8299 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:32:46.335839 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-d4ff72bd-0b65-401f-b0ce-534285d30df8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d638d98e53e4d81a0b18218e3acdc25'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:46.336347 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-d4ff72bd-0b65-401f-b0ce-534285d30df8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d638d98e53e4d81a0b18218e3acdc25'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:46.336547 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ea0f7a-7859-429a-b322-342d57a8af49 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:46.336937 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ea0f7a-7859-429a-b322-342d57a8af49 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:46.337129 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-d4ff72bd-0b65-401f-b0ce-534285d30df8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d638d98e53e4d81a0b18218e3acdc25'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:46.337238 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ea0f7a-7859-429a-b322-342d57a8af49 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:46.349447 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1129/2283] 10.222.0.102 () {54 vars in 966 bytes} [Thu Sep 1 17:32:46 2022] GET /identity => generated 271 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:32:46.352394 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1155/2284] 10.222.0.102 () {58 vars in 1229 bytes} [Thu Sep 1 17:32:46 2022] DELETE /identity/v3/users/9d638d98e53e4d81a0b18218e3acdc25 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:32:46.353934 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c176c029-ed76-491b-8b6f-e9e641046c26 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:46.354671 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c176c029-ed76-491b-8b6f-e9e641046c26 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:46.355100 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c176c029-ed76-491b-8b6f-e9e641046c26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:46.356704 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85dc751-f229-4574-9b2d-637c548f422f None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:46.356990 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85dc751-f229-4574-9b2d-637c548f422f None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:46.357271 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85dc751-f229-4574-9b2d-637c548f422f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:46.379201 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-c176c029-ed76-491b-8b6f-e9e641046c26 None None] MFA Rules not processed for user `134ef6039cef468bae0e3f3be79b8073`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:32:46.380717 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-f85dc751-f229-4574-9b2d-637c548f422f None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:32:46.380954 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-c176c029-ed76-491b-8b6f-e9e641046c26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:46.381959 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-f85dc751-f229-4574-9b2d-637c548f422f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:46.449232 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1156/2285] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:32:46 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:32:46.457567 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1130/2286] 10.222.0.102 () {58 vars in 1063 bytes} [Thu Sep 1 17:32:46 2022] POST /identity/v3/auth/tokens => generated 3318 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:32:46.461178 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85dc751-f229-4574-9b2d-637c548f422f None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:46.462747 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-f85dc751-f229-4574-9b2d-637c548f422f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:46.465780 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c176c029-ed76-491b-8b6f-e9e641046c26 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:46.477641 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c176c029-ed76-491b-8b6f-e9e641046c26 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:46.483265 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d51344c-99a7-4931-a8fc-f40abdb63eb0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '134ef6039cef468bae0e3f3be79b8073', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:46.483510 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d51344c-99a7-4931-a8fc-f40abdb63eb0 service placement] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:46.483795 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d51344c-99a7-4931-a8fc-f40abdb63eb0 service placement] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:46.484069 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d51344c-99a7-4931-a8fc-f40abdb63eb0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:46.486292 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-2d51344c-99a7-4931-a8fc-f40abdb63eb0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:46.498280 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d51344c-99a7-4931-a8fc-f40abdb63eb0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:46.499853 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d51344c-99a7-4931-a8fc-f40abdb63eb0 service placement] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:46.514191 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1131/2287] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:32:46 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 17:32:46.536481 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d51344c-99a7-4931-a8fc-f40abdb63eb0 service placement] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:46.537981 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-2d51344c-99a7-4931-a8fc-f40abdb63eb0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:46.555249 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85dc751-f229-4574-9b2d-637c548f422f None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:46.555704 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d51344c-99a7-4931-a8fc-f40abdb63eb0 service placement] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:46.560817 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4d9bf0-a610-4072-ada3-4dd8e032dcb4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:46.561536 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4d9bf0-a610-4072-ada3-4dd8e032dcb4 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:46.561819 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4d9bf0-a610-4072-ada3-4dd8e032dcb4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:46.562122 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4d9bf0-a610-4072-ada3-4dd8e032dcb4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:46.562671 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f2351a-31e6-4635-9e71-41eb318ac37b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:46.563320 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f2351a-31e6-4635-9e71-41eb318ac37b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:46.563603 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f2351a-31e6-4635-9e71-41eb318ac37b admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:46.563912 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f2351a-31e6-4635-9e71-41eb318ac37b admin admin] PATH_INFO: `/v3/projects/1f232c720f5a4108a46e79700be67099` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:46.565484 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f2351a-31e6-4635-9e71-41eb318ac37b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f232c720f5a4108a46e79700be67099)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:46.567476 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f2351a-31e6-4635-9e71-41eb318ac37b admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:46.571376 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:32:46.575080 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4d9bf0-a610-4072-ada3-4dd8e032dcb4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:46.577259 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4d9bf0-a610-4072-ada3-4dd8e032dcb4 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:46.593946 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1157/2288] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:32:46 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 17:32:46.611623 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:32:46.626954 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-21f2351a-31e6-4635-9e71-41eb318ac37b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f232c720f5a4108a46e79700be67099', 'request_id': 'req-21f2351a-31e6-4635-9e71-41eb318ac37b'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:46.632531 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-21f2351a-31e6-4635-9e71-41eb318ac37b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f232c720f5a4108a46e79700be67099', 'request_id': 'req-21f2351a-31e6-4635-9e71-41eb318ac37b'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:46.640009 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-21f2351a-31e6-4635-9e71-41eb318ac37b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f232c720f5a4108a46e79700be67099', 'request_id': 'req-21f2351a-31e6-4635-9e71-41eb318ac37b'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:46.652415 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-21f2351a-31e6-4635-9e71-41eb318ac37b admin admin] The token cache is being invalidate because project 1f232c720f5a4108a46e79700be67099 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:32:46.652715 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-21f2351a-31e6-4635-9e71-41eb318ac37b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:46.653486 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-21f2351a-31e6-4635-9e71-41eb318ac37b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:46.655002 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1132/2289] 10.222.0.102 () {58 vars in 1238 bytes} [Thu Sep 1 17:32:46 2022] DELETE /identity/v3/projects/1f232c720f5a4108a46e79700be67099 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:32:46.659942 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35505f82-cae4-45fb-97ef-c1a9b56747ac None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:46.660237 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35505f82-cae4-45fb-97ef-c1a9b56747ac None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:46.660616 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35505f82-cae4-45fb-97ef-c1a9b56747ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:46.683689 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-35505f82-cae4-45fb-97ef-c1a9b56747ac None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:32:46.684894 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-35505f82-cae4-45fb-97ef-c1a9b56747ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:46.751420 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1158/2290] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:32:46 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:32:46.759902 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae74902-ca53-40d3-90df-aa7933275e10 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:46.759902 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae74902-ca53-40d3-90df-aa7933275e10 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:46.759902 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae74902-ca53-40d3-90df-aa7933275e10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:46.783537 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-2ae74902-ca53-40d3-90df-aa7933275e10 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:32:46.784744 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-2ae74902-ca53-40d3-90df-aa7933275e10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:46.794080 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1133/2291] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:32:46 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:32:46.798114 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35505f82-cae4-45fb-97ef-c1a9b56747ac None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:46.808586 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35505f82-cae4-45fb-97ef-c1a9b56747ac None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:46.815014 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c081ba53-ba3b-427d-bb0a-dd8a53b2eadd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:46.815634 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c081ba53-ba3b-427d-bb0a-dd8a53b2eadd admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:46.816296 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c081ba53-ba3b-427d-bb0a-dd8a53b2eadd admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:46.816960 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c081ba53-ba3b-427d-bb0a-dd8a53b2eadd admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:46.817219 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c081ba53-ba3b-427d-bb0a-dd8a53b2eadd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:32:46.817557 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c081ba53-ba3b-427d-bb0a-dd8a53b2eadd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:46.819184 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c081ba53-ba3b-427d-bb0a-dd8a53b2eadd admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:46.833346 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1159/2292] 10.222.0.102 () {58 vars in 1158 bytes} [Thu Sep 1 17:32:46 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:32:46.836563 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae74902-ca53-40d3-90df-aa7933275e10 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:46.854159 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae74902-ca53-40d3-90df-aa7933275e10 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:46.862936 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec8c77d-5a64-4960-b663-318d35c93daf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:46.863565 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec8c77d-5a64-4960-b663-318d35c93daf admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:46.863845 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec8c77d-5a64-4960-b663-318d35c93daf admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:46.864699 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec8c77d-5a64-4960-b663-318d35c93daf admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:46.865611 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec8c77d-5a64-4960-b663-318d35c93daf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:46.868482 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec8c77d-5a64-4960-b663-318d35c93daf admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:46.901169 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e4bd90-eabe-496e-b2db-27508bcda955 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:46.901604 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e4bd90-eabe-496e-b2db-27508bcda955 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:46.901946 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e4bd90-eabe-496e-b2db-27508bcda955 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:46.908187 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1134/2293] 10.222.0.102 () {60 vars in 1158 bytes} [Thu Sep 1 17:32:46 2022] POST /identity/v3/projects => generated 376 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:32:46.912423 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec8c77d-5a64-4960-b663-318d35c93daf admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:46.939926 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-84e4bd90-eabe-496e-b2db-27508bcda955 None None] MFA Rules not processed for user `b54b95db13d84fc4b026fdd119cb7ec5`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:32:46.941105 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-84e4bd90-eabe-496e-b2db-27508bcda955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:46.989932 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec8c77d-5a64-4960-b663-318d35c93daf admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:46.995765 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64d203c-b4f1-4c04-9da7-38624557e216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:46.996472 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64d203c-b4f1-4c04-9da7-38624557e216 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:46.996817 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64d203c-b4f1-4c04-9da7-38624557e216 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:46.997511 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64d203c-b4f1-4c04-9da7-38624557e216 admin admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:46.997877 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64d203c-b4f1-4c04-9da7-38624557e216 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:46.999872 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64d203c-b4f1-4c04-9da7-38624557e216 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:47.010315 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1160/2294] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:32:46 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:32:47.011329 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e4bd90-eabe-496e-b2db-27508bcda955 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:47.012950 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-84e4bd90-eabe-496e-b2db-27508bcda955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:47.025855 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1135/2295] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:32:46 2022] POST /identity/v3/users => generated 289 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:32:47.027294 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64d203c-b4f1-4c04-9da7-38624557e216 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:47.030639 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e4bd90-eabe-496e-b2db-27508bcda955 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:47.036592 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36058af-9b18-4af7-af6f-a974d5f49383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:47.037338 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36058af-9b18-4af7-af6f-a974d5f49383 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:47.037700 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36058af-9b18-4af7-af6f-a974d5f49383 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:47.038047 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36058af-9b18-4af7-af6f-a974d5f49383 admin admin] PATH_INFO: `/v3/projects/0a69f0aa0eff449f9802515e7a5960fd` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:47.038208 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64d203c-b4f1-4c04-9da7-38624557e216 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:47.039548 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36058af-9b18-4af7-af6f-a974d5f49383 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a69f0aa0eff449f9802515e7a5960fd)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:47.042606 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36058af-9b18-4af7-af6f-a974d5f49383 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:47.043602 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681b0c75-39e8-441a-8c97-a026b030334b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:47.044276 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681b0c75-39e8-441a-8c97-a026b030334b admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:47.044947 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681b0c75-39e8-441a-8c97-a026b030334b admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:47.045038 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681b0c75-39e8-441a-8c97-a026b030334b admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:47.045894 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681b0c75-39e8-441a-8c97-a026b030334b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:47.047482 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681b0c75-39e8-441a-8c97-a026b030334b admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:47.058534 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1136/2296] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:32:47 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 17:32:47.062229 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681b0c75-39e8-441a-8c97-a026b030334b admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:47.072209 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681b0c75-39e8-441a-8c97-a026b030334b admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:47.077069 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6258450-da41-4b59-a67c-6d104cedef5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:47.077859 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6258450-da41-4b59-a67c-6d104cedef5d admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:47.077965 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6258450-da41-4b59-a67c-6d104cedef5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:47.078212 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6258450-da41-4b59-a67c-6d104cedef5d admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:47.079042 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6258450-da41-4b59-a67c-6d104cedef5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:47.080575 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6258450-da41-4b59-a67c-6d104cedef5d admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:47.092197 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1137/2297] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:32:47 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 17:32:47.096634 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6258450-da41-4b59-a67c-6d104cedef5d admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:47.102485 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-e36058af-9b18-4af7-af6f-a974d5f49383 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a69f0aa0eff449f9802515e7a5960fd', 'request_id': 'req-e36058af-9b18-4af7-af6f-a974d5f49383'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:47.108014 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6258450-da41-4b59-a67c-6d104cedef5d admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:47.112267 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-e36058af-9b18-4af7-af6f-a974d5f49383 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a69f0aa0eff449f9802515e7a5960fd', 'request_id': 'req-e36058af-9b18-4af7-af6f-a974d5f49383'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:47.119674 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-e36058af-9b18-4af7-af6f-a974d5f49383 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a69f0aa0eff449f9802515e7a5960fd', 'request_id': 'req-e36058af-9b18-4af7-af6f-a974d5f49383'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:47.132087 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-e36058af-9b18-4af7-af6f-a974d5f49383 admin admin] The token cache is being invalidate because project 0a69f0aa0eff449f9802515e7a5960fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:32:47.132426 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-e36058af-9b18-4af7-af6f-a974d5f49383 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:47.133100 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-e36058af-9b18-4af7-af6f-a974d5f49383 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:47.135101 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1161/2298] 10.222.0.102 () {58 vars in 1238 bytes} [Thu Sep 1 17:32:47 2022] DELETE /identity/v3/projects/0a69f0aa0eff449f9802515e7a5960fd => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:32:47.142220 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36058af-9b18-4af7-af6f-a974d5f49383 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:47.143571 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-e36058af-9b18-4af7-af6f-a974d5f49383 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:47.158806 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:32:47.168193 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeffad8-49ac-4b75-a9c0-151dbaab7cae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:47.168865 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeffad8-49ac-4b75-a9c0-151dbaab7cae admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:47.169151 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeffad8-49ac-4b75-a9c0-151dbaab7cae admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:47.169468 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeffad8-49ac-4b75-a9c0-151dbaab7cae admin admin] PATH_INFO: `/v3/projects/19bb1cbe315441258828677997dd6fdf/users/ae0aab3e503249d49ff9473b4990ff63/roles/b75cb360925e4ba5a65f14965c103ffd` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:47.171871 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:32:47.191621 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aeffad8-49ac-4b75-a9c0-151dbaab7cae admin admin] RBAC: Authorizing `identity:create_grant(project_id=19bb1cbe315441258828677997dd6fdf, user_id=ae0aab3e503249d49ff9473b4990ff63, role_id=b75cb360925e4ba5a65f14965c103ffd)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:47.194623 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aeffad8-49ac-4b75-a9c0-151dbaab7cae admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:47.215122 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1138/2299] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:32:47 2022] PUT /identity/v3/projects/19bb1cbe315441258828677997dd6fdf/users/ae0aab3e503249d49ff9473b4990ff63/roles/b75cb360925e4ba5a65f14965c103ffd => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:32:47.218690 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb8e4f5-8553-4686-a51f-7054c0893f39 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:47.219007 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb8e4f5-8553-4686-a51f-7054c0893f39 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:47.219254 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb8e4f5-8553-4686-a51f-7054c0893f39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:47.226378 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36058af-9b18-4af7-af6f-a974d5f49383 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:47.243882 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-fcb8e4f5-8553-4686-a51f-7054c0893f39 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:32:47.246165 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-fcb8e4f5-8553-4686-a51f-7054c0893f39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:47.295902 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fad10f1-f04c-4d09-8ff1-321674af88b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:47.297106 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fad10f1-f04c-4d09-8ff1-321674af88b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:47.297762 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fad10f1-f04c-4d09-8ff1-321674af88b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:47.298193 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fad10f1-f04c-4d09-8ff1-321674af88b3 admin admin] PATH_INFO: `/v3/users/2d1808a56ada4a7ab22cf1c39aa154f9` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:47.299682 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fad10f1-f04c-4d09-8ff1-321674af88b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d1808a56ada4a7ab22cf1c39aa154f9)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:47.302317 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fad10f1-f04c-4d09-8ff1-321674af88b3 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:47.311860 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1139/2300] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:32:47 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:32:47.322446 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb8e4f5-8553-4686-a51f-7054c0893f39 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:47.324547 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-fcb8e4f5-8553-4686-a51f-7054c0893f39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:47.358061 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-3fad10f1-f04c-4d09-8ff1-321674af88b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d1808a56ada4a7ab22cf1c39aa154f9'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:47.372558 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-3fad10f1-f04c-4d09-8ff1-321674af88b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d1808a56ada4a7ab22cf1c39aa154f9'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:47.382128 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-3fad10f1-f04c-4d09-8ff1-321674af88b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d1808a56ada4a7ab22cf1c39aa154f9'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:47.383119 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-3fad10f1-f04c-4d09-8ff1-321674af88b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d1808a56ada4a7ab22cf1c39aa154f9'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:47.384036 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-3fad10f1-f04c-4d09-8ff1-321674af88b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d1808a56ada4a7ab22cf1c39aa154f9'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:47.402974 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1162/2301] 10.222.0.102 () {58 vars in 1229 bytes} [Thu Sep 1 17:32:47 2022] DELETE /identity/v3/users/2d1808a56ada4a7ab22cf1c39aa154f9 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:32:47.533189 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb8e4f5-8553-4686-a51f-7054c0893f39 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:47.600277 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f533ffa-30b8-4dab-bbae-b6f245af493e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:47.601070 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f533ffa-30b8-4dab-bbae-b6f245af493e service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:47.601389 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f533ffa-30b8-4dab-bbae-b6f245af493e service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:47.601669 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f533ffa-30b8-4dab-bbae-b6f245af493e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:47.604123 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-8f533ffa-30b8-4dab-bbae-b6f245af493e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:47.710296 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:32:47.728599 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f533ffa-30b8-4dab-bbae-b6f245af493e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:47.730322 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f533ffa-30b8-4dab-bbae-b6f245af493e service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:47.742503 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fad10f1-f04c-4d09-8ff1-321674af88b3 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:47.744311 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-3fad10f1-f04c-4d09-8ff1-321674af88b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:47.745289 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1140/2302] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:32:47 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 17:32:47.784834 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fad10f1-f04c-4d09-8ff1-321674af88b3 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:47.791420 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5a2112-22be-4c14-bf2d-f88e7f39627d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:47.792343 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5a2112-22be-4c14-bf2d-f88e7f39627d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:47.792721 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5a2112-22be-4c14-bf2d-f88e7f39627d admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:47.793487 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5a2112-22be-4c14-bf2d-f88e7f39627d admin admin] PATH_INFO: `/v3/projects/53bbf3bbe4c04adaa9d540ecdf769b32` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:47.794754 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5a2112-22be-4c14-bf2d-f88e7f39627d admin admin] RBAC: Authorizing `identity:delete_project(project_id=53bbf3bbe4c04adaa9d540ecdf769b32)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:47.796734 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5a2112-22be-4c14-bf2d-f88e7f39627d admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:47.857852 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:32:47.878018 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9b5a2112-22be-4c14-bf2d-f88e7f39627d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53bbf3bbe4c04adaa9d540ecdf769b32', 'request_id': 'req-9b5a2112-22be-4c14-bf2d-f88e7f39627d'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:47.887733 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9b5a2112-22be-4c14-bf2d-f88e7f39627d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53bbf3bbe4c04adaa9d540ecdf769b32', 'request_id': 'req-9b5a2112-22be-4c14-bf2d-f88e7f39627d'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:47.897185 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9b5a2112-22be-4c14-bf2d-f88e7f39627d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53bbf3bbe4c04adaa9d540ecdf769b32', 'request_id': 'req-9b5a2112-22be-4c14-bf2d-f88e7f39627d'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:47.912808 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9b5a2112-22be-4c14-bf2d-f88e7f39627d admin admin] The token cache is being invalidate because project 53bbf3bbe4c04adaa9d540ecdf769b32 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:32:47.913500 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9b5a2112-22be-4c14-bf2d-f88e7f39627d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:47.914442 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9b5a2112-22be-4c14-bf2d-f88e7f39627d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:47.916655 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1163/2303] 10.222.0.102 () {58 vars in 1238 bytes} [Thu Sep 1 17:32:47 2022] DELETE /identity/v3/projects/53bbf3bbe4c04adaa9d540ecdf769b32 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:32:47.921043 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f533ffa-30b8-4dab-bbae-b6f245af493e service neutron] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:47.932146 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f533ffa-30b8-4dab-bbae-b6f245af493e service neutron] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:48.013066 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847fada6-24f8-49b5-8ad1-cca8280f252c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:48.013843 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847fada6-24f8-49b5-8ad1-cca8280f252c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:48.014130 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847fada6-24f8-49b5-8ad1-cca8280f252c admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:48.014407 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847fada6-24f8-49b5-8ad1-cca8280f252c admin admin] PATH_INFO: `/v3/users/33bb714ea00947138bb5a9c29c03f7c6` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:48.015584 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847fada6-24f8-49b5-8ad1-cca8280f252c admin admin] RBAC: Authorizing `identity:delete_user(user_id=33bb714ea00947138bb5a9c29c03f7c6)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:48.017912 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847fada6-24f8-49b5-8ad1-cca8280f252c admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:48.087353 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-847fada6-24f8-49b5-8ad1-cca8280f252c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33bb714ea00947138bb5a9c29c03f7c6'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:48.090446 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-847fada6-24f8-49b5-8ad1-cca8280f252c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33bb714ea00947138bb5a9c29c03f7c6'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:48.107707 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-847fada6-24f8-49b5-8ad1-cca8280f252c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33bb714ea00947138bb5a9c29c03f7c6'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:48.117174 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-847fada6-24f8-49b5-8ad1-cca8280f252c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33bb714ea00947138bb5a9c29c03f7c6'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:48.117828 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-847fada6-24f8-49b5-8ad1-cca8280f252c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33bb714ea00947138bb5a9c29c03f7c6'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:48.133526 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1141/2304] 10.222.0.102 () {58 vars in 1229 bytes} [Thu Sep 1 17:32:47 2022] DELETE /identity/v3/users/33bb714ea00947138bb5a9c29c03f7c6 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:32:48.465494 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5a2112-22be-4c14-bf2d-f88e7f39627d admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:48.467531 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-9b5a2112-22be-4c14-bf2d-f88e7f39627d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:48.490522 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5a2112-22be-4c14-bf2d-f88e7f39627d admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:48.583363 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f497cce-cd15-4ccd-82b6-2241f81dc3f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:48.583363 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f497cce-cd15-4ccd-82b6-2241f81dc3f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:48.583363 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f497cce-cd15-4ccd-82b6-2241f81dc3f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:48.584122 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f497cce-cd15-4ccd-82b6-2241f81dc3f6 admin admin] PATH_INFO: `/v3/projects/60d4b903a6ef4febbfbf650bc85be686` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:48.585839 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f497cce-cd15-4ccd-82b6-2241f81dc3f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=60d4b903a6ef4febbfbf650bc85be686)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:48.587799 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f497cce-cd15-4ccd-82b6-2241f81dc3f6 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:48.669279 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-8f497cce-cd15-4ccd-82b6-2241f81dc3f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60d4b903a6ef4febbfbf650bc85be686', 'request_id': 'req-8f497cce-cd15-4ccd-82b6-2241f81dc3f6'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:48.680217 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-8f497cce-cd15-4ccd-82b6-2241f81dc3f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60d4b903a6ef4febbfbf650bc85be686', 'request_id': 'req-8f497cce-cd15-4ccd-82b6-2241f81dc3f6'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:48.701583 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-8f497cce-cd15-4ccd-82b6-2241f81dc3f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60d4b903a6ef4febbfbf650bc85be686', 'request_id': 'req-8f497cce-cd15-4ccd-82b6-2241f81dc3f6'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:48.737040 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-8f497cce-cd15-4ccd-82b6-2241f81dc3f6 admin admin] The token cache is being invalidate because project 60d4b903a6ef4febbfbf650bc85be686 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:32:48.737506 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-8f497cce-cd15-4ccd-82b6-2241f81dc3f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:48.738898 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-8f497cce-cd15-4ccd-82b6-2241f81dc3f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:48.742166 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1164/2305] 10.222.0.102 () {58 vars in 1238 bytes} [Thu Sep 1 17:32:48 2022] DELETE /identity/v3/projects/60d4b903a6ef4febbfbf650bc85be686 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:32:48.748649 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad53e4ef-bf9e-4f8c-9f0f-ab04d5aa5ea2 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:48.748990 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad53e4ef-bf9e-4f8c-9f0f-ab04d5aa5ea2 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:48.749271 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad53e4ef-bf9e-4f8c-9f0f-ab04d5aa5ea2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:48.783964 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-ad53e4ef-bf9e-4f8c-9f0f-ab04d5aa5ea2 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:32:48.785603 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-ad53e4ef-bf9e-4f8c-9f0f-ab04d5aa5ea2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:48.873718 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1142/2306] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:32:48 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:32:48.881244 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0c8aba-e025-40a8-a344-8502e49d9f3a None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:48.881611 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0c8aba-e025-40a8-a344-8502e49d9f3a None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:48.881898 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0c8aba-e025-40a8-a344-8502e49d9f3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:48.904987 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-ea0c8aba-e025-40a8-a344-8502e49d9f3a None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:32:48.905856 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-ea0c8aba-e025-40a8-a344-8502e49d9f3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:48.914412 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1165/2307] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:32:48 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:32:48.919136 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad53e4ef-bf9e-4f8c-9f0f-ab04d5aa5ea2 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:48.929661 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad53e4ef-bf9e-4f8c-9f0f-ab04d5aa5ea2 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:48.934393 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2917801f-896e-43eb-8357-b6d8b04dbe09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:48.935139 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2917801f-896e-43eb-8357-b6d8b04dbe09 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:48.935705 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2917801f-896e-43eb-8357-b6d8b04dbe09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:48.935705 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2917801f-896e-43eb-8357-b6d8b04dbe09 admin admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:48.936533 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2917801f-896e-43eb-8357-b6d8b04dbe09 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:32:48.936848 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2917801f-896e-43eb-8357-b6d8b04dbe09 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:48.938603 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2917801f-896e-43eb-8357-b6d8b04dbe09 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:48.941839 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:32:48.958650 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1143/2308] 10.222.0.102 () {58 vars in 1158 bytes} [Thu Sep 1 17:32:48 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:32:48.960623 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0c8aba-e025-40a8-a344-8502e49d9f3a None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:48.972657 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0c8aba-e025-40a8-a344-8502e49d9f3a None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:48.977636 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bc762b-6da9-4138-84e4-b5c92a7fac5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:48.978421 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bc762b-6da9-4138-84e4-b5c92a7fac5a admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:48.978536 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bc762b-6da9-4138-84e4-b5c92a7fac5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:48.978785 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bc762b-6da9-4138-84e4-b5c92a7fac5a admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:48.979421 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bc762b-6da9-4138-84e4-b5c92a7fac5a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:48.981982 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bc762b-6da9-4138-84e4-b5c92a7fac5a admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:49.026923 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1166/2309] 10.222.0.102 () {60 vars in 1158 bytes} [Thu Sep 1 17:32:48 2022] POST /identity/v3/projects => generated 386 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:32:49.031231 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2917801f-896e-43eb-8357-b6d8b04dbe09 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:49.043610 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2917801f-896e-43eb-8357-b6d8b04dbe09 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:49.122514 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a42920-0282-49ea-861e-50e9b0a3e500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:49.123952 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a42920-0282-49ea-861e-50e9b0a3e500 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:49.124425 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a42920-0282-49ea-861e-50e9b0a3e500 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:49.124880 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a42920-0282-49ea-861e-50e9b0a3e500 admin admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:49.126267 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a42920-0282-49ea-861e-50e9b0a3e500 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:49.128457 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a42920-0282-49ea-861e-50e9b0a3e500 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:49.155512 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1144/2310] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:32:49 2022] POST /identity/v3/users => generated 294 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:32:49.159596 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bc762b-6da9-4138-84e4-b5c92a7fac5a admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:49.173731 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bc762b-6da9-4138-84e4-b5c92a7fac5a admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:49.179046 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851bf2d4-5c9b-4691-84f1-02f8c9eeb816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:49.179805 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851bf2d4-5c9b-4691-84f1-02f8c9eeb816 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:49.180192 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851bf2d4-5c9b-4691-84f1-02f8c9eeb816 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:49.180568 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851bf2d4-5c9b-4691-84f1-02f8c9eeb816 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:49.181408 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851bf2d4-5c9b-4691-84f1-02f8c9eeb816 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:49.183126 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851bf2d4-5c9b-4691-84f1-02f8c9eeb816 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:49.196684 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1167/2311] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:32:49 2022] GET /identity/v3/roles => generated 1427 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 17:32:49.201583 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a42920-0282-49ea-861e-50e9b0a3e500 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:49.213189 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a42920-0282-49ea-861e-50e9b0a3e500 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:49.220347 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222c5122-8e17-4d00-8618-f21d00cb9432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:49.221409 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222c5122-8e17-4d00-8618-f21d00cb9432 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:49.221904 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222c5122-8e17-4d00-8618-f21d00cb9432 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:49.222444 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222c5122-8e17-4d00-8618-f21d00cb9432 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:49.223392 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222c5122-8e17-4d00-8618-f21d00cb9432 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:49.225903 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222c5122-8e17-4d00-8618-f21d00cb9432 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:49.238909 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1145/2312] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:32:49 2022] GET /identity/v3/roles => generated 1427 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:32:49.243405 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851bf2d4-5c9b-4691-84f1-02f8c9eeb816 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:49.255866 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851bf2d4-5c9b-4691-84f1-02f8c9eeb816 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:49.267076 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0952f34-8d5a-47e4-b5ed-4b8375f3d38c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:49.267901 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0952f34-8d5a-47e4-b5ed-4b8375f3d38c admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:49.268692 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0952f34-8d5a-47e4-b5ed-4b8375f3d38c admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:49.268692 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0952f34-8d5a-47e4-b5ed-4b8375f3d38c admin admin] PATH_INFO: `/v3/projects/0c1d8e734a7c4e3ca696ba01d510a8c3/users/4ef92885e13b4314b0b479965fe0a5b0/roles/b75cb360925e4ba5a65f14965c103ffd` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:49.282914 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:32:49.318492 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0952f34-8d5a-47e4-b5ed-4b8375f3d38c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c1d8e734a7c4e3ca696ba01d510a8c3, user_id=4ef92885e13b4314b0b479965fe0a5b0, role_id=b75cb360925e4ba5a65f14965c103ffd)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:49.324651 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0952f34-8d5a-47e4-b5ed-4b8375f3d38c admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:49.355105 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1168/2313] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:32:49 2022] PUT /identity/v3/projects/0c1d8e734a7c4e3ca696ba01d510a8c3/users/4ef92885e13b4314b0b479965fe0a5b0/roles/b75cb360925e4ba5a65f14965c103ffd => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:32:49.366382 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffa8216-0c6d-4059-a42c-8cba7040e8c8 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:49.366890 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffa8216-0c6d-4059-a42c-8cba7040e8c8 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:49.367542 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffa8216-0c6d-4059-a42c-8cba7040e8c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:49.404673 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-3ffa8216-0c6d-4059-a42c-8cba7040e8c8 None None] MFA Rules not processed for user `4ef92885e13b4314b0b479965fe0a5b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:32:49.406414 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-3ffa8216-0c6d-4059-a42c-8cba7040e8c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:49.515670 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1146/2314] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:32:49 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:32:49.520211 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0952f34-8d5a-47e4-b5ed-4b8375f3d38c admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:49.646311 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0952f34-8d5a-47e4-b5ed-4b8375f3d38c admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:49.651617 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8ec50d-9fa6-40e4-b220-38e403efd63f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:49.652246 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8ec50d-9fa6-40e4-b220-38e403efd63f admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:49.652530 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8ec50d-9fa6-40e4-b220-38e403efd63f admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:49.652806 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8ec50d-9fa6-40e4-b220-38e403efd63f admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:49.653518 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8ec50d-9fa6-40e4-b220-38e403efd63f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:49.655472 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8ec50d-9fa6-40e4-b220-38e403efd63f admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:49.699639 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1169/2315] 10.222.0.102 () {60 vars in 1158 bytes} [Thu Sep 1 17:32:49 2022] POST /identity/v3/projects => generated 384 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:32:49.703705 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffa8216-0c6d-4059-a42c-8cba7040e8c8 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:49.716991 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffa8216-0c6d-4059-a42c-8cba7040e8c8 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:49.821190 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1674357a-8890-4e8b-817d-f763420b6819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:49.821920 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1674357a-8890-4e8b-817d-f763420b6819 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:49.822229 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1674357a-8890-4e8b-817d-f763420b6819 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:49.822464 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1674357a-8890-4e8b-817d-f763420b6819 admin admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:49.823133 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1674357a-8890-4e8b-817d-f763420b6819 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:49.825973 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1674357a-8890-4e8b-817d-f763420b6819 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:49.837982 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:32:49.864433 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1147/2316] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:32:49 2022] POST /identity/v3/users => generated 293 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:32:49.866981 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8ec50d-9fa6-40e4-b220-38e403efd63f admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:49.888875 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8ec50d-9fa6-40e4-b220-38e403efd63f admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:49.897460 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4530430d-3847-4ec7-982c-360b7b3a9620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:49.898390 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4530430d-3847-4ec7-982c-360b7b3a9620 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:49.898842 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4530430d-3847-4ec7-982c-360b7b3a9620 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:49.899217 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4530430d-3847-4ec7-982c-360b7b3a9620 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:49.900179 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4530430d-3847-4ec7-982c-360b7b3a9620 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:49.902562 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4530430d-3847-4ec7-982c-360b7b3a9620 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:49.917641 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1170/2317] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:32:49 2022] GET /identity/v3/roles => generated 1427 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:32:49.921958 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1674357a-8890-4e8b-817d-f763420b6819 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:49.934558 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1674357a-8890-4e8b-817d-f763420b6819 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:49.940108 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308cdeef-12f0-4713-bd58-e28e95664339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:49.940716 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308cdeef-12f0-4713-bd58-e28e95664339 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:49.940991 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308cdeef-12f0-4713-bd58-e28e95664339 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:49.941267 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308cdeef-12f0-4713-bd58-e28e95664339 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:49.942091 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308cdeef-12f0-4713-bd58-e28e95664339 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:49.943689 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308cdeef-12f0-4713-bd58-e28e95664339 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:49.954586 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1148/2318] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:32:49 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:32:49.959602 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4530430d-3847-4ec7-982c-360b7b3a9620 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:49.970692 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4530430d-3847-4ec7-982c-360b7b3a9620 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:49.976298 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fd05b6-3dbe-40bc-8a12-b1a4e1e0c767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:49.977025 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fd05b6-3dbe-40bc-8a12-b1a4e1e0c767 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:49.977334 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fd05b6-3dbe-40bc-8a12-b1a4e1e0c767 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:49.977607 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fd05b6-3dbe-40bc-8a12-b1a4e1e0c767 admin admin] PATH_INFO: `/v3/projects/f4d7afa787c74ca4bfe5f0f2501282b9/users/15670fd9dc024fb9af9395cb66b9f217/roles/b75cb360925e4ba5a65f14965c103ffd` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:49.995772 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fd05b6-3dbe-40bc-8a12-b1a4e1e0c767 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4d7afa787c74ca4bfe5f0f2501282b9, user_id=15670fd9dc024fb9af9395cb66b9f217, role_id=b75cb360925e4ba5a65f14965c103ffd)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:49.998287 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fd05b6-3dbe-40bc-8a12-b1a4e1e0c767 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:50.025518 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1171/2319] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:32:49 2022] PUT /identity/v3/projects/f4d7afa787c74ca4bfe5f0f2501282b9/users/15670fd9dc024fb9af9395cb66b9f217/roles/b75cb360925e4ba5a65f14965c103ffd => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:32:50.027275 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02834ae-bd25-45ab-96f1-6ef32a8a898c None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:50.027563 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02834ae-bd25-45ab-96f1-6ef32a8a898c None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:50.027833 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02834ae-bd25-45ab-96f1-6ef32a8a898c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:50.056795 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-d02834ae-bd25-45ab-96f1-6ef32a8a898c None None] MFA Rules not processed for user `15670fd9dc024fb9af9395cb66b9f217`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:32:50.058681 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-d02834ae-bd25-45ab-96f1-6ef32a8a898c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:50.148572 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1149/2320] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:32:50 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:32:50.150319 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fd05b6-3dbe-40bc-8a12-b1a4e1e0c767 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:50.299987 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fd05b6-3dbe-40bc-8a12-b1a4e1e0c767 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:50.312236 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502907c5-4212-4637-9f62-cdb7ae1efe39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:50.313040 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502907c5-4212-4637-9f62-cdb7ae1efe39 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:50.313454 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502907c5-4212-4637-9f62-cdb7ae1efe39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:50.313922 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502907c5-4212-4637-9f62-cdb7ae1efe39 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:50.314699 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502907c5-4212-4637-9f62-cdb7ae1efe39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:50.316784 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502907c5-4212-4637-9f62-cdb7ae1efe39 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:50.373957 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02834ae-bd25-45ab-96f1-6ef32a8a898c None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:50.374161 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1172/2321] 10.222.0.102 () {60 vars in 1158 bytes} [Thu Sep 1 17:32:50 2022] POST /identity/v3/projects => generated 384 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:32:50.391734 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02834ae-bd25-45ab-96f1-6ef32a8a898c None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:50.529534 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ed0221-8e1d-41c4-8d29-5e576c16751e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:50.530249 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ed0221-8e1d-41c4-8d29-5e576c16751e admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:50.530527 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ed0221-8e1d-41c4-8d29-5e576c16751e admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:50.530793 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ed0221-8e1d-41c4-8d29-5e576c16751e admin admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:50.534284 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ed0221-8e1d-41c4-8d29-5e576c16751e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:50.538559 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ed0221-8e1d-41c4-8d29-5e576c16751e admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:50.574476 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1150/2322] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:32:50 2022] POST /identity/v3/users => generated 299 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:32:50.578473 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502907c5-4212-4637-9f62-cdb7ae1efe39 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:50.596893 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502907c5-4212-4637-9f62-cdb7ae1efe39 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:50.602608 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e43b284-49f4-4d5e-9bb1-dbb5d57d833b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:50.603554 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e43b284-49f4-4d5e-9bb1-dbb5d57d833b admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:50.603954 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e43b284-49f4-4d5e-9bb1-dbb5d57d833b admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:50.604355 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e43b284-49f4-4d5e-9bb1-dbb5d57d833b admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:50.605138 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e43b284-49f4-4d5e-9bb1-dbb5d57d833b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:50.607020 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e43b284-49f4-4d5e-9bb1-dbb5d57d833b admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:50.622548 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1173/2323] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:32:50 2022] GET /identity/v3/roles => generated 1427 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:32:50.623769 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ed0221-8e1d-41c4-8d29-5e576c16751e admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:50.634320 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ed0221-8e1d-41c4-8d29-5e576c16751e admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:50.641091 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50536a1a-1dd9-41c5-9ddd-17613fa9472b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:50.642011 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50536a1a-1dd9-41c5-9ddd-17613fa9472b admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:50.642502 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50536a1a-1dd9-41c5-9ddd-17613fa9472b admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:50.642982 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50536a1a-1dd9-41c5-9ddd-17613fa9472b admin admin] PATH_INFO: `/v3/projects/b6edfea9501e41f585a367751e8fa4d5/users/b91c4c41991d4ef594c5d4ffd92a16be/roles/36577a6f46d642d4b81805ede9872490` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:50.670229 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50536a1a-1dd9-41c5-9ddd-17613fa9472b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6edfea9501e41f585a367751e8fa4d5, user_id=b91c4c41991d4ef594c5d4ffd92a16be, role_id=36577a6f46d642d4b81805ede9872490)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:50.674233 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50536a1a-1dd9-41c5-9ddd-17613fa9472b admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:50.701789 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1151/2324] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:32:50 2022] PUT /identity/v3/projects/b6edfea9501e41f585a367751e8fa4d5/users/b91c4c41991d4ef594c5d4ffd92a16be/roles/36577a6f46d642d4b81805ede9872490 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:32:50.702218 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e43b284-49f4-4d5e-9bb1-dbb5d57d833b admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:50.721429 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e43b284-49f4-4d5e-9bb1-dbb5d57d833b admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:50.739989 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:32:50.834076 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56aac454-6275-4b65-9327-41d51f915544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:50.834888 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aac454-6275-4b65-9327-41d51f915544 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:50.835179 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aac454-6275-4b65-9327-41d51f915544 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:50.835426 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aac454-6275-4b65-9327-41d51f915544 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:50.839719 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56aac454-6275-4b65-9327-41d51f915544 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:50.845165 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56aac454-6275-4b65-9327-41d51f915544 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:50.859351 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1174/2325] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:32:50 2022] GET /identity/v3/roles => generated 1427 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 17:32:50.862328 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50536a1a-1dd9-41c5-9ddd-17613fa9472b admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:50.882235 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50536a1a-1dd9-41c5-9ddd-17613fa9472b admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:50.891962 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbb50b8-4ce0-4ec5-bf5d-5fa3e467e5bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:50.892787 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbb50b8-4ce0-4ec5-bf5d-5fa3e467e5bd admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:50.893072 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbb50b8-4ce0-4ec5-bf5d-5fa3e467e5bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:50.893468 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbb50b8-4ce0-4ec5-bf5d-5fa3e467e5bd admin admin] PATH_INFO: `/v3/projects/b6edfea9501e41f585a367751e8fa4d5/users/b91c4c41991d4ef594c5d4ffd92a16be/roles/36577a6f46d642d4b81805ede9872490` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:50.896655 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbb50b8-4ce0-4ec5-bf5d-5fa3e467e5bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6edfea9501e41f585a367751e8fa4d5, user_id=b91c4c41991d4ef594c5d4ffd92a16be, role_id=36577a6f46d642d4b81805ede9872490)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:50.902945 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbb50b8-4ce0-4ec5-bf5d-5fa3e467e5bd admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:50.916316 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1152/2326] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:32:50 2022] PUT /identity/v3/projects/b6edfea9501e41f585a367751e8fa4d5/users/b91c4c41991d4ef594c5d4ffd92a16be/roles/36577a6f46d642d4b81805ede9872490 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:32:50.927282 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24ee7f9-8cc5-4fd5-853b-401d7be6eaa6 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:50.927591 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24ee7f9-8cc5-4fd5-853b-401d7be6eaa6 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:50.927858 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24ee7f9-8cc5-4fd5-853b-401d7be6eaa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:50.964250 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-d24ee7f9-8cc5-4fd5-853b-401d7be6eaa6 None None] MFA Rules not processed for user `b91c4c41991d4ef594c5d4ffd92a16be`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:32:50.967094 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-d24ee7f9-8cc5-4fd5-853b-401d7be6eaa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:51.077850 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1175/2327] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:32:50 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:32:51.091183 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbb50b8-4ce0-4ec5-bf5d-5fa3e467e5bd admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:51.093206 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-bfbb50b8-4ce0-4ec5-bf5d-5fa3e467e5bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:51.149449 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:32:51.207946 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbb50b8-4ce0-4ec5-bf5d-5fa3e467e5bd admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:51.216718 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a147849-0b5a-4033-9156-9c2d2f8ce0ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '523ed589872a45a09b6a7675a7fb0b75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:51.217901 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a147849-0b5a-4033-9156-9c2d2f8ce0ae service cinder] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:51.218326 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a147849-0b5a-4033-9156-9c2d2f8ce0ae service cinder] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:51.218852 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a147849-0b5a-4033-9156-9c2d2f8ce0ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:51.338033 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a147849-0b5a-4033-9156-9c2d2f8ce0ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:51.340664 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a147849-0b5a-4033-9156-9c2d2f8ce0ae service cinder] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:51.366351 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1153/2328] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:32:51 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:32:53.092275 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e178dff2-cf62-43fb-877f-97552595a454 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:53.093134 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e178dff2-cf62-43fb-877f-97552595a454 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:53.093517 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e178dff2-cf62-43fb-877f-97552595a454 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:53.132101 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-e178dff2-cf62-43fb-877f-97552595a454 None None] MFA Rules not processed for user `ae0aab3e503249d49ff9473b4990ff63`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:32:53.133553 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-e178dff2-cf62-43fb-877f-97552595a454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:53.259473 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1176/2329] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:32:53 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:32:53.270225 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a147849-0b5a-4033-9156-9c2d2f8ce0ae service cinder] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:53.272440 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-2a147849-0b5a-4033-9156-9c2d2f8ce0ae service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:53.371611 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a147849-0b5a-4033-9156-9c2d2f8ce0ae service cinder] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:53.378387 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6e932a-8aea-4d09-bd48-4fcddae4a0fe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f8293316baf4774b62fc11d55d50865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:53.379382 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6e932a-8aea-4d09-bd48-4fcddae4a0fe service nova] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:53.379873 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6e932a-8aea-4d09-bd48-4fcddae4a0fe service nova] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:53.380358 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6e932a-8aea-4d09-bd48-4fcddae4a0fe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:53.392056 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6e932a-8aea-4d09-bd48-4fcddae4a0fe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:53.394830 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6e932a-8aea-4d09-bd48-4fcddae4a0fe service nova] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:53.407446 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1154/2330] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:32:53 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:32:53.548209 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e178dff2-cf62-43fb-877f-97552595a454 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:53.550878 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-e178dff2-cf62-43fb-877f-97552595a454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:53.584451 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:32:53.640282 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e178dff2-cf62-43fb-877f-97552595a454 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:53.646043 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfad26e-38f5-4049-9d39-75018f728190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:53.646751 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfad26e-38f5-4049-9d39-75018f728190 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:53.647036 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfad26e-38f5-4049-9d39-75018f728190 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:53.647322 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfad26e-38f5-4049-9d39-75018f728190 admin admin] PATH_INFO: `/v3/users/25e2c68661dc421ca27eccdbea6a7f93` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:53.648469 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfad26e-38f5-4049-9d39-75018f728190 admin admin] RBAC: Authorizing `identity:delete_user(user_id=25e2c68661dc421ca27eccdbea6a7f93)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:53.650492 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfad26e-38f5-4049-9d39-75018f728190 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:53.729473 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-3bfad26e-38f5-4049-9d39-75018f728190 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25e2c68661dc421ca27eccdbea6a7f93'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:53.758389 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-3bfad26e-38f5-4049-9d39-75018f728190 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25e2c68661dc421ca27eccdbea6a7f93'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:53.782422 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-3bfad26e-38f5-4049-9d39-75018f728190 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25e2c68661dc421ca27eccdbea6a7f93'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:53.783827 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-3bfad26e-38f5-4049-9d39-75018f728190 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25e2c68661dc421ca27eccdbea6a7f93'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:53.785701 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-3bfad26e-38f5-4049-9d39-75018f728190 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25e2c68661dc421ca27eccdbea6a7f93'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:53.812310 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1177/2331] 10.222.0.102 () {58 vars in 1229 bytes} [Thu Sep 1 17:32:53 2022] DELETE /identity/v3/users/25e2c68661dc421ca27eccdbea6a7f93 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:32:53.813869 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4ab0ec-db75-4244-8ce2-75569a1068d7 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:53.814316 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4ab0ec-db75-4244-8ce2-75569a1068d7 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:53.814897 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4ab0ec-db75-4244-8ce2-75569a1068d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:53.842293 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-de4ab0ec-db75-4244-8ce2-75569a1068d7 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:32:53.844042 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-de4ab0ec-db75-4244-8ce2-75569a1068d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:53.947996 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1155/2332] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:32:53 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:32:53.958240 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfad26e-38f5-4049-9d39-75018f728190 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:53.959642 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-3bfad26e-38f5-4049-9d39-75018f728190 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:54.039989 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfad26e-38f5-4049-9d39-75018f728190 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:54.046857 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c5e6ca-0ac6-4a27-a78f-042efe78e04e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:54.047729 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c5e6ca-0ac6-4a27-a78f-042efe78e04e service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:54.048484 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c5e6ca-0ac6-4a27-a78f-042efe78e04e service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:54.048919 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c5e6ca-0ac6-4a27-a78f-042efe78e04e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:54.062508 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c5e6ca-0ac6-4a27-a78f-042efe78e04e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:54.064575 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c5e6ca-0ac6-4a27-a78f-042efe78e04e service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:54.077861 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1178/2333] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:32:53 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 17:32:54.181441 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4ab0ec-db75-4244-8ce2-75569a1068d7 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:54.182993 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-de4ab0ec-db75-4244-8ce2-75569a1068d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:54.191007 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:32:54.260926 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4ab0ec-db75-4244-8ce2-75569a1068d7 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:54.264954 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b54f6f-71a9-44d2-bb01-09063777fb94 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '523ed589872a45a09b6a7675a7fb0b75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:54.265628 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b54f6f-71a9-44d2-bb01-09063777fb94 service cinder] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:54.265885 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b54f6f-71a9-44d2-bb01-09063777fb94 service cinder] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:54.266362 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b54f6f-71a9-44d2-bb01-09063777fb94 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:54.268003 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-c5b54f6f-71a9-44d2-bb01-09063777fb94 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:54.351367 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b54f6f-71a9-44d2-bb01-09063777fb94 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:54.353474 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b54f6f-71a9-44d2-bb01-09063777fb94 service cinder] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:54.364881 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1156/2334] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:32:54 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:32:54.723252 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c5e6ca-0ac6-4a27-a78f-042efe78e04e service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:54.725264 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-b9c5e6ca-0ac6-4a27-a78f-042efe78e04e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:54.746933 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c5e6ca-0ac6-4a27-a78f-042efe78e04e service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:54.752441 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9b4e35-a1a7-4cd7-b0e3-00a1fd76d207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:54.754390 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9b4e35-a1a7-4cd7-b0e3-00a1fd76d207 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:54.754679 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9b4e35-a1a7-4cd7-b0e3-00a1fd76d207 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:54.754996 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9b4e35-a1a7-4cd7-b0e3-00a1fd76d207 admin admin] PATH_INFO: `/v3/projects/74914f5328ea4a9ebde30423869af79f` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:54.757785 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9b4e35-a1a7-4cd7-b0e3-00a1fd76d207 admin admin] RBAC: Authorizing `identity:delete_project(project_id=74914f5328ea4a9ebde30423869af79f)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:54.760233 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9b4e35-a1a7-4cd7-b0e3-00a1fd76d207 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:54.835835 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:32:54.847300 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-3f9b4e35-a1a7-4cd7-b0e3-00a1fd76d207 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74914f5328ea4a9ebde30423869af79f', 'request_id': 'req-3f9b4e35-a1a7-4cd7-b0e3-00a1fd76d207'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:54.860056 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-3f9b4e35-a1a7-4cd7-b0e3-00a1fd76d207 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74914f5328ea4a9ebde30423869af79f', 'request_id': 'req-3f9b4e35-a1a7-4cd7-b0e3-00a1fd76d207'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:54.867271 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-3f9b4e35-a1a7-4cd7-b0e3-00a1fd76d207 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74914f5328ea4a9ebde30423869af79f', 'request_id': 'req-3f9b4e35-a1a7-4cd7-b0e3-00a1fd76d207'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:54.881295 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-3f9b4e35-a1a7-4cd7-b0e3-00a1fd76d207 admin admin] The token cache is being invalidate because project 74914f5328ea4a9ebde30423869af79f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:32:54.881638 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-3f9b4e35-a1a7-4cd7-b0e3-00a1fd76d207 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:54.882268 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-3f9b4e35-a1a7-4cd7-b0e3-00a1fd76d207 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:32:54.883885 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1179/2335] 10.222.0.102 () {58 vars in 1238 bytes} [Thu Sep 1 17:32:54 2022] DELETE /identity/v3/projects/74914f5328ea4a9ebde30423869af79f => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:32:54.890188 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b621418a-7955-4632-82b5-2ff0bf7f289c None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:54.890626 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b621418a-7955-4632-82b5-2ff0bf7f289c None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:54.890937 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b621418a-7955-4632-82b5-2ff0bf7f289c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:54.915149 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-b621418a-7955-4632-82b5-2ff0bf7f289c None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:32:54.917580 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-b621418a-7955-4632-82b5-2ff0bf7f289c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:55.006472 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1157/2336] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:32:54 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:32:55.014958 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d749f21-c570-4dab-bd18-feb0802191c9 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:55.015248 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d749f21-c570-4dab-bd18-feb0802191c9 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:55.015519 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d749f21-c570-4dab-bd18-feb0802191c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:55.035656 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-2d749f21-c570-4dab-bd18-feb0802191c9 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:32:55.036876 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-2d749f21-c570-4dab-bd18-feb0802191c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:55.046922 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1180/2337] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:32:55 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:32:55.051816 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b621418a-7955-4632-82b5-2ff0bf7f289c None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:55.063759 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b621418a-7955-4632-82b5-2ff0bf7f289c None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:55.078823 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a49f427-74df-4b87-8684-8effbeee9838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:55.078823 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a49f427-74df-4b87-8684-8effbeee9838 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:55.078823 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a49f427-74df-4b87-8684-8effbeee9838 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:55.078823 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a49f427-74df-4b87-8684-8effbeee9838 admin admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:55.079129 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a49f427-74df-4b87-8684-8effbeee9838 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:32:55.079129 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a49f427-74df-4b87-8684-8effbeee9838 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:55.079129 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a49f427-74df-4b87-8684-8effbeee9838 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:55.095925 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1158/2338] 10.222.0.102 () {58 vars in 1158 bytes} [Thu Sep 1 17:32:55 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:32:55.097044 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d749f21-c570-4dab-bd18-feb0802191c9 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:55.111829 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d749f21-c570-4dab-bd18-feb0802191c9 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:55.118678 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ef52ab-0c0f-47fc-9da5-831169eb3dd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:55.119399 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ef52ab-0c0f-47fc-9da5-831169eb3dd7 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:55.120332 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ef52ab-0c0f-47fc-9da5-831169eb3dd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:55.120701 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ef52ab-0c0f-47fc-9da5-831169eb3dd7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:55.121559 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ef52ab-0c0f-47fc-9da5-831169eb3dd7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:55.124154 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ef52ab-0c0f-47fc-9da5-831169eb3dd7 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:55.160366 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a49f427-74df-4b87-8684-8effbeee9838 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:55.162139 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-2a49f427-74df-4b87-8684-8effbeee9838 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:55.163781 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1181/2339] 10.222.0.102 () {60 vars in 1158 bytes} [Thu Sep 1 17:32:55 2022] POST /identity/v3/projects => generated 384 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:32:55.168455 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ef52ab-0c0f-47fc-9da5-831169eb3dd7 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:55.240338 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:32:55.252050 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a49f427-74df-4b87-8684-8effbeee9838 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:55.265232 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ef52ab-0c0f-47fc-9da5-831169eb3dd7 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:55.271562 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493cfdc0-e3bb-4133-a2cc-e1e2e2a7ddb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:55.272234 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493cfdc0-e3bb-4133-a2cc-e1e2e2a7ddb0 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:55.272503 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493cfdc0-e3bb-4133-a2cc-e1e2e2a7ddb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:55.272767 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493cfdc0-e3bb-4133-a2cc-e1e2e2a7ddb0 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:55.273476 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493cfdc0-e3bb-4133-a2cc-e1e2e2a7ddb0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:55.275617 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493cfdc0-e3bb-4133-a2cc-e1e2e2a7ddb0 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:55.303256 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1182/2340] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:32:55 2022] POST /identity/v3/users => generated 293 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:32:55.307748 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493cfdc0-e3bb-4133-a2cc-e1e2e2a7ddb0 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:55.317239 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eaa020-65e2-4fd2-9929-79022f1ee48f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '523ed589872a45a09b6a7675a7fb0b75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:55.318170 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eaa020-65e2-4fd2-9929-79022f1ee48f service cinder] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:55.319016 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eaa020-65e2-4fd2-9929-79022f1ee48f service cinder] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:55.319443 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eaa020-65e2-4fd2-9929-79022f1ee48f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:55.322149 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-c9eaa020-65e2-4fd2-9929-79022f1ee48f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:55.322303 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493cfdc0-e3bb-4133-a2cc-e1e2e2a7ddb0 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:55.328059 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db61aac7-2605-4ebf-8df5-6dd193d1302d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:55.329130 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db61aac7-2605-4ebf-8df5-6dd193d1302d admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:55.329130 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db61aac7-2605-4ebf-8df5-6dd193d1302d admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:55.329377 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db61aac7-2605-4ebf-8df5-6dd193d1302d admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:55.329872 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db61aac7-2605-4ebf-8df5-6dd193d1302d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:55.332291 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db61aac7-2605-4ebf-8df5-6dd193d1302d admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:55.343875 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1183/2341] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:32:55 2022] GET /identity/v3/roles => generated 1427 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:32:55.347816 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db61aac7-2605-4ebf-8df5-6dd193d1302d admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:55.359028 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db61aac7-2605-4ebf-8df5-6dd193d1302d admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:55.367005 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ece539-e60f-4c60-b150-bfd97ecd6b9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:55.367637 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ece539-e60f-4c60-b150-bfd97ecd6b9a admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:55.368101 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ece539-e60f-4c60-b150-bfd97ecd6b9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:55.368379 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ece539-e60f-4c60-b150-bfd97ecd6b9a admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:55.369237 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ece539-e60f-4c60-b150-bfd97ecd6b9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:55.371332 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ece539-e60f-4c60-b150-bfd97ecd6b9a admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:55.385710 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1184/2342] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:32:55 2022] GET /identity/v3/roles => generated 1427 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:32:55.386989 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a6d8f5-f7cd-43a3-8ab9-15fd9e52c620 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:55.387272 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a6d8f5-f7cd-43a3-8ab9-15fd9e52c620 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:55.387543 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a6d8f5-f7cd-43a3-8ab9-15fd9e52c620 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:55.406960 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-f2a6d8f5-f7cd-43a3-8ab9-15fd9e52c620 None None] MFA Rules not processed for user `ae0aab3e503249d49ff9473b4990ff63`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:32:55.408155 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-f2a6d8f5-f7cd-43a3-8ab9-15fd9e52c620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:55.418363 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9eaa020-65e2-4fd2-9929-79022f1ee48f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:55.419786 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9eaa020-65e2-4fd2-9929-79022f1ee48f service cinder] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:55.429233 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1159/2343] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:32:55 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:32:55.430587 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eaa020-65e2-4fd2-9929-79022f1ee48f service cinder] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:55.449999 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eaa020-65e2-4fd2-9929-79022f1ee48f service cinder] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:55.455921 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beef300f-7c67-43f3-859f-bd5665c43b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:55.456786 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beef300f-7c67-43f3-859f-bd5665c43b05 admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:55.457065 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beef300f-7c67-43f3-859f-bd5665c43b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:55.457362 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beef300f-7c67-43f3-859f-bd5665c43b05 admin admin] PATH_INFO: `/v3/projects/1c5b780c53444adf9c22202c0aa3d339/users/574437ec44d64cbe86ca5e61f3ec4f1a/roles/b75cb360925e4ba5a65f14965c103ffd` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:55.477090 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beef300f-7c67-43f3-859f-bd5665c43b05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c5b780c53444adf9c22202c0aa3d339, user_id=574437ec44d64cbe86ca5e61f3ec4f1a, role_id=b75cb360925e4ba5a65f14965c103ffd)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:55.480551 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beef300f-7c67-43f3-859f-bd5665c43b05 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:55.485274 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1185/2344] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:32:55 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:32:55.509353 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1160/2345] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:32:55 2022] PUT /identity/v3/projects/1c5b780c53444adf9c22202c0aa3d339/users/574437ec44d64cbe86ca5e61f3ec4f1a/roles/b75cb360925e4ba5a65f14965c103ffd => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:32:55.517097 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1098d77-f078-41b6-83ac-132fc276674f None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:55.517623 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1098d77-f078-41b6-83ac-132fc276674f None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:55.518093 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1098d77-f078-41b6-83ac-132fc276674f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:55.543787 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:32:55.559362 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-d1098d77-f078-41b6-83ac-132fc276674f None None] MFA Rules not processed for user `574437ec44d64cbe86ca5e61f3ec4f1a`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:32:55.561301 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-d1098d77-f078-41b6-83ac-132fc276674f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:55.694472 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1186/2346] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:32:55 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:32:55.708084 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beef300f-7c67-43f3-859f-bd5665c43b05 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:55.787024 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beef300f-7c67-43f3-859f-bd5665c43b05 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:55.791195 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da2b52a-56b3-49ab-99cf-46c40dfeebe7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '523ed589872a45a09b6a7675a7fb0b75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:55.792040 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da2b52a-56b3-49ab-99cf-46c40dfeebe7 service cinder] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:55.792536 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da2b52a-56b3-49ab-99cf-46c40dfeebe7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:55.793499 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da2b52a-56b3-49ab-99cf-46c40dfeebe7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:55.806977 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da2b52a-56b3-49ab-99cf-46c40dfeebe7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:55.808719 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da2b52a-56b3-49ab-99cf-46c40dfeebe7 service cinder] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:55.821526 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1161/2347] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:32:55 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:33:04.782529 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1098d77-f078-41b6-83ac-132fc276674f None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:33:04.788630 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-d1098d77-f078-41b6-83ac-132fc276674f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:33:04.871756 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1098d77-f078-41b6-83ac-132fc276674f None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:33:04.877690 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c54c93-f047-419b-9bbc-829fd37d2aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:33:04.878333 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c54c93-f047-419b-9bbc-829fd37d2aef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:33:04.878617 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c54c93-f047-419b-9bbc-829fd37d2aef admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:33:04.878893 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c54c93-f047-419b-9bbc-829fd37d2aef admin admin] PATH_INFO: `/v3/users/4ef92885e13b4314b0b479965fe0a5b0` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:33:04.880071 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c54c93-f047-419b-9bbc-829fd37d2aef admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ef92885e13b4314b0b479965fe0a5b0)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:33:04.882080 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c54c93-f047-419b-9bbc-829fd37d2aef admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:33:04.930459 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-29c54c93-f047-419b-9bbc-829fd37d2aef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ef92885e13b4314b0b479965fe0a5b0'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:33:04.944792 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-29c54c93-f047-419b-9bbc-829fd37d2aef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ef92885e13b4314b0b479965fe0a5b0'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:33:04.952870 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-29c54c93-f047-419b-9bbc-829fd37d2aef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ef92885e13b4314b0b479965fe0a5b0'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:33:04.953784 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-29c54c93-f047-419b-9bbc-829fd37d2aef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ef92885e13b4314b0b479965fe0a5b0'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:33:04.954400 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-29c54c93-f047-419b-9bbc-829fd37d2aef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ef92885e13b4314b0b479965fe0a5b0'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:33:04.967538 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1187/2348] 10.222.0.102 () {58 vars in 1229 bytes} [Thu Sep 1 17:33:04 2022] DELETE /identity/v3/users/4ef92885e13b4314b0b479965fe0a5b0 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:33:04.972265 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43700099-8d28-4552-b425-df9c6a9d4147 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:33:04.972722 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43700099-8d28-4552-b425-df9c6a9d4147 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:33:04.973122 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43700099-8d28-4552-b425-df9c6a9d4147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:33:04.992937 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-43700099-8d28-4552-b425-df9c6a9d4147 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:33:04.993865 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-43700099-8d28-4552-b425-df9c6a9d4147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:33:05.025218 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:33:05.067976 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1162/2349] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:33:04 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:33:05.077042 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c54c93-f047-419b-9bbc-829fd37d2aef admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:33:05.079009 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-29c54c93-f047-419b-9bbc-829fd37d2aef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:33:05.117498 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:33:05.165806 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c54c93-f047-419b-9bbc-829fd37d2aef admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:33:05.170604 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f137459-e904-44c9-a806-5df75da9f19f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:33:05.171226 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f137459-e904-44c9-a806-5df75da9f19f service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:33:05.171506 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f137459-e904-44c9-a806-5df75da9f19f service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:33:05.171778 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f137459-e904-44c9-a806-5df75da9f19f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:33:05.186370 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f137459-e904-44c9-a806-5df75da9f19f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:33:05.188264 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f137459-e904-44c9-a806-5df75da9f19f service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:33:05.203371 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1188/2350] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:33:05 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:33:05.572033 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43700099-8d28-4552-b425-df9c6a9d4147 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:33:05.573618 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-43700099-8d28-4552-b425-df9c6a9d4147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:33:05.588572 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43700099-8d28-4552-b425-df9c6a9d4147 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:33:05.593779 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6483e72e-226c-4c25-af57-b87cff406cd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:33:05.594682 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6483e72e-226c-4c25-af57-b87cff406cd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:33:05.594968 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6483e72e-226c-4c25-af57-b87cff406cd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:33:05.595244 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6483e72e-226c-4c25-af57-b87cff406cd6 admin admin] PATH_INFO: `/v3/projects/0c1d8e734a7c4e3ca696ba01d510a8c3` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:33:05.596612 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6483e72e-226c-4c25-af57-b87cff406cd6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c1d8e734a7c4e3ca696ba01d510a8c3)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:33:05.598715 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6483e72e-226c-4c25-af57-b87cff406cd6 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:33:05.654964 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-6483e72e-226c-4c25-af57-b87cff406cd6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c1d8e734a7c4e3ca696ba01d510a8c3', 'request_id': 'req-6483e72e-226c-4c25-af57-b87cff406cd6'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:33:05.660854 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-6483e72e-226c-4c25-af57-b87cff406cd6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c1d8e734a7c4e3ca696ba01d510a8c3', 'request_id': 'req-6483e72e-226c-4c25-af57-b87cff406cd6'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:33:05.672646 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-6483e72e-226c-4c25-af57-b87cff406cd6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c1d8e734a7c4e3ca696ba01d510a8c3', 'request_id': 'req-6483e72e-226c-4c25-af57-b87cff406cd6'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:33:05.686298 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-6483e72e-226c-4c25-af57-b87cff406cd6 admin admin] The token cache is being invalidate because project 0c1d8e734a7c4e3ca696ba01d510a8c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:33:05.686722 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-6483e72e-226c-4c25-af57-b87cff406cd6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:33:05.687228 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-6483e72e-226c-4c25-af57-b87cff406cd6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:33:05.688731 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1163/2351] 10.222.0.102 () {58 vars in 1238 bytes} [Thu Sep 1 17:33:05 2022] DELETE /identity/v3/projects/0c1d8e734a7c4e3ca696ba01d510a8c3 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:33:05.692845 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f137459-e904-44c9-a806-5df75da9f19f service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:33:05.706927 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f137459-e904-44c9-a806-5df75da9f19f service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:33:05.788319 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a08b14a-ed8c-40e3-b06f-3ce4b21ecb23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:33:05.789709 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a08b14a-ed8c-40e3-b06f-3ce4b21ecb23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:33:05.790133 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a08b14a-ed8c-40e3-b06f-3ce4b21ecb23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:33:05.790682 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a08b14a-ed8c-40e3-b06f-3ce4b21ecb23 admin admin] PATH_INFO: `/v3/users/15670fd9dc024fb9af9395cb66b9f217` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:33:05.792838 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a08b14a-ed8c-40e3-b06f-3ce4b21ecb23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=15670fd9dc024fb9af9395cb66b9f217)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:33:05.794851 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a08b14a-ed8c-40e3-b06f-3ce4b21ecb23 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:33:05.869814 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-6a08b14a-ed8c-40e3-b06f-3ce4b21ecb23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15670fd9dc024fb9af9395cb66b9f217'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:33:05.887583 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-6a08b14a-ed8c-40e3-b06f-3ce4b21ecb23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15670fd9dc024fb9af9395cb66b9f217'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:33:05.896121 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-6a08b14a-ed8c-40e3-b06f-3ce4b21ecb23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15670fd9dc024fb9af9395cb66b9f217'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:33:05.896748 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-6a08b14a-ed8c-40e3-b06f-3ce4b21ecb23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15670fd9dc024fb9af9395cb66b9f217'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:33:05.897400 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-6a08b14a-ed8c-40e3-b06f-3ce4b21ecb23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15670fd9dc024fb9af9395cb66b9f217'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:33:05.914290 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1189/2352] 10.222.0.102 () {58 vars in 1229 bytes} [Thu Sep 1 17:33:05 2022] DELETE /identity/v3/users/15670fd9dc024fb9af9395cb66b9f217 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:33:06.233767 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6483e72e-226c-4c25-af57-b87cff406cd6 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:33:06.235552 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-6483e72e-226c-4c25-af57-b87cff406cd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:33:06.251705 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6483e72e-226c-4c25-af57-b87cff406cd6 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:33:06.310050 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c656326a-7a22-42b1-b443-a15ba0b5ddba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:33:06.310712 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c656326a-7a22-42b1-b443-a15ba0b5ddba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:33:06.310993 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c656326a-7a22-42b1-b443-a15ba0b5ddba admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:33:06.311265 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c656326a-7a22-42b1-b443-a15ba0b5ddba admin admin] PATH_INFO: `/v3/projects/f4d7afa787c74ca4bfe5f0f2501282b9` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:33:06.312429 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c656326a-7a22-42b1-b443-a15ba0b5ddba admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4d7afa787c74ca4bfe5f0f2501282b9)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:33:06.314371 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c656326a-7a22-42b1-b443-a15ba0b5ddba admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:33:06.363861 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-c656326a-7a22-42b1-b443-a15ba0b5ddba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4d7afa787c74ca4bfe5f0f2501282b9', 'request_id': 'req-c656326a-7a22-42b1-b443-a15ba0b5ddba'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:33:06.368407 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:33:06.370105 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-c656326a-7a22-42b1-b443-a15ba0b5ddba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4d7afa787c74ca4bfe5f0f2501282b9', 'request_id': 'req-c656326a-7a22-42b1-b443-a15ba0b5ddba'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:33:06.378043 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-c656326a-7a22-42b1-b443-a15ba0b5ddba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4d7afa787c74ca4bfe5f0f2501282b9', 'request_id': 'req-c656326a-7a22-42b1-b443-a15ba0b5ddba'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:33:06.390212 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-c656326a-7a22-42b1-b443-a15ba0b5ddba admin admin] The token cache is being invalidate because project f4d7afa787c74ca4bfe5f0f2501282b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:33:06.390501 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-c656326a-7a22-42b1-b443-a15ba0b5ddba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:33:06.391142 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-c656326a-7a22-42b1-b443-a15ba0b5ddba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:33:06.392623 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1164/2353] 10.222.0.102 () {58 vars in 1238 bytes} [Thu Sep 1 17:33:06 2022] DELETE /identity/v3/projects/f4d7afa787c74ca4bfe5f0f2501282b9 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:33:06.396622 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a08b14a-ed8c-40e3-b06f-3ce4b21ecb23 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:33:06.398190 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-6a08b14a-ed8c-40e3-b06f-3ce4b21ecb23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:33:06.412023 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a08b14a-ed8c-40e3-b06f-3ce4b21ecb23 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:33:06.453726 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:33:06.471980 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6bda86-fd78-4ff2-894d-cb90af9787f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:33:06.472609 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6bda86-fd78-4ff2-894d-cb90af9787f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:33:06.472890 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6bda86-fd78-4ff2-894d-cb90af9787f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:33:06.473165 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6bda86-fd78-4ff2-894d-cb90af9787f3 admin admin] PATH_INFO: `/v3/users/b91c4c41991d4ef594c5d4ffd92a16be` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:33:06.474323 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6bda86-fd78-4ff2-894d-cb90af9787f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b91c4c41991d4ef594c5d4ffd92a16be)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:33:06.476234 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6bda86-fd78-4ff2-894d-cb90af9787f3 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:33:06.522955 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-7d6bda86-fd78-4ff2-894d-cb90af9787f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b91c4c41991d4ef594c5d4ffd92a16be'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:33:06.536832 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-7d6bda86-fd78-4ff2-894d-cb90af9787f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b91c4c41991d4ef594c5d4ffd92a16be'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:33:06.547356 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-7d6bda86-fd78-4ff2-894d-cb90af9787f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b91c4c41991d4ef594c5d4ffd92a16be'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:33:06.548222 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-7d6bda86-fd78-4ff2-894d-cb90af9787f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b91c4c41991d4ef594c5d4ffd92a16be'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:33:06.549490 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-7d6bda86-fd78-4ff2-894d-cb90af9787f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b91c4c41991d4ef594c5d4ffd92a16be'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:33:06.563781 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1190/2354] 10.222.0.102 () {58 vars in 1229 bytes} [Thu Sep 1 17:33:06 2022] DELETE /identity/v3/users/b91c4c41991d4ef594c5d4ffd92a16be => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:33:06.935615 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c656326a-7a22-42b1-b443-a15ba0b5ddba admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:33:06.937381 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-c656326a-7a22-42b1-b443-a15ba0b5ddba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:33:06.952946 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c656326a-7a22-42b1-b443-a15ba0b5ddba admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:33:07.017709 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945de3fe-c530-427a-87fd-b3630d2447e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:33:07.018364 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945de3fe-c530-427a-87fd-b3630d2447e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:33:07.018638 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945de3fe-c530-427a-87fd-b3630d2447e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:33:07.018905 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945de3fe-c530-427a-87fd-b3630d2447e1 admin admin] PATH_INFO: `/v3/projects/b6edfea9501e41f585a367751e8fa4d5` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:33:07.020002 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945de3fe-c530-427a-87fd-b3630d2447e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6edfea9501e41f585a367751e8fa4d5)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:33:07.021935 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945de3fe-c530-427a-87fd-b3630d2447e1 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:33:07.068643 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-945de3fe-c530-427a-87fd-b3630d2447e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6edfea9501e41f585a367751e8fa4d5', 'request_id': 'req-945de3fe-c530-427a-87fd-b3630d2447e1'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:33:07.074684 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-945de3fe-c530-427a-87fd-b3630d2447e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6edfea9501e41f585a367751e8fa4d5', 'request_id': 'req-945de3fe-c530-427a-87fd-b3630d2447e1'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:33:07.083235 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-945de3fe-c530-427a-87fd-b3630d2447e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6edfea9501e41f585a367751e8fa4d5', 'request_id': 'req-945de3fe-c530-427a-87fd-b3630d2447e1'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:33:07.095310 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-945de3fe-c530-427a-87fd-b3630d2447e1 admin admin] The token cache is being invalidate because project b6edfea9501e41f585a367751e8fa4d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:33:07.095595 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-945de3fe-c530-427a-87fd-b3630d2447e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:33:07.096361 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-945de3fe-c530-427a-87fd-b3630d2447e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:33:07.097866 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1165/2355] 10.222.0.102 () {58 vars in 1238 bytes} [Thu Sep 1 17:33:06 2022] DELETE /identity/v3/projects/b6edfea9501e41f585a367751e8fa4d5 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:33:07.103165 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aac01da-48c4-4a21-b414-cca1ce432299 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:33:07.103638 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aac01da-48c4-4a21-b414-cca1ce432299 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:33:07.103753 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aac01da-48c4-4a21-b414-cca1ce432299 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:33:07.122987 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-6aac01da-48c4-4a21-b414-cca1ce432299 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:33:07.124184 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-6aac01da-48c4-4a21-b414-cca1ce432299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:33:07.201334 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1191/2356] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:33:07 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:33:07.255815 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2679a9a-e961-4ab6-adfb-ef7ce3536e68 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:33:07.256372 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2679a9a-e961-4ab6-adfb-ef7ce3536e68 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:33:07.256372 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2679a9a-e961-4ab6-adfb-ef7ce3536e68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:33:07.277195 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-b2679a9a-e961-4ab6-adfb-ef7ce3536e68 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:33:07.278498 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-b2679a9a-e961-4ab6-adfb-ef7ce3536e68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:33:07.287006 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1166/2357] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:33:07 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:33:07.291633 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aac01da-48c4-4a21-b414-cca1ce432299 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:33:07.301796 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aac01da-48c4-4a21-b414-cca1ce432299 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:33:07.307040 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89600664-cd78-4f4a-81f3-14ac120598f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:33:07.307668 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89600664-cd78-4f4a-81f3-14ac120598f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:33:07.307946 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89600664-cd78-4f4a-81f3-14ac120598f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:33:07.308216 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89600664-cd78-4f4a-81f3-14ac120598f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:33:07.308859 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89600664-cd78-4f4a-81f3-14ac120598f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:33:07.309190 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89600664-cd78-4f4a-81f3-14ac120598f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:33:07.310944 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89600664-cd78-4f4a-81f3-14ac120598f3 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:33:07.323213 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1192/2358] 10.222.0.102 () {58 vars in 1158 bytes} [Thu Sep 1 17:33:07 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:33:07.327774 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2679a9a-e961-4ab6-adfb-ef7ce3536e68 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:33:07.336726 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2679a9a-e961-4ab6-adfb-ef7ce3536e68 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:33:07.341345 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ab20c0-64fa-49ac-87cb-b20814d6e29f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:33:07.341922 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ab20c0-64fa-49ac-87cb-b20814d6e29f admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:33:07.342243 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ab20c0-64fa-49ac-87cb-b20814d6e29f admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:33:07.342547 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ab20c0-64fa-49ac-87cb-b20814d6e29f admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:33:07.343230 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ab20c0-64fa-49ac-87cb-b20814d6e29f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:33:07.345560 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ab20c0-64fa-49ac-87cb-b20814d6e29f admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:33:07.372797 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1167/2359] 10.222.0.102 () {60 vars in 1158 bytes} [Thu Sep 1 17:33:07 2022] POST /identity/v3/projects => generated 392 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:33:07.376870 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89600664-cd78-4f4a-81f3-14ac120598f3 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:33:07.387520 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89600664-cd78-4f4a-81f3-14ac120598f3 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:33:07.430630 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:33:07.445574 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8571a881-ec9e-4fd1-88b7-b0cc8e443545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:33:07.446174 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8571a881-ec9e-4fd1-88b7-b0cc8e443545 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:33:07.446454 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8571a881-ec9e-4fd1-88b7-b0cc8e443545 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:33:07.446748 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8571a881-ec9e-4fd1-88b7-b0cc8e443545 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:33:07.448005 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8571a881-ec9e-4fd1-88b7-b0cc8e443545 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:33:07.449953 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8571a881-ec9e-4fd1-88b7-b0cc8e443545 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:33:07.470969 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1193/2360] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:33:07 2022] POST /identity/v3/users => generated 297 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:33:07.474997 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ab20c0-64fa-49ac-87cb-b20814d6e29f admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:33:07.485162 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ab20c0-64fa-49ac-87cb-b20814d6e29f admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:33:07.489754 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d60a7f-7cda-4b0f-996d-20567499fee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:33:07.490346 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d60a7f-7cda-4b0f-996d-20567499fee1 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:33:07.490622 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d60a7f-7cda-4b0f-996d-20567499fee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:33:07.490890 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d60a7f-7cda-4b0f-996d-20567499fee1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:33:07.491529 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d60a7f-7cda-4b0f-996d-20567499fee1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:33:07.493091 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d60a7f-7cda-4b0f-996d-20567499fee1 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:33:07.503486 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1168/2361] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:33:07 2022] GET /identity/v3/roles => generated 1427 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:33:07.507518 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8571a881-ec9e-4fd1-88b7-b0cc8e443545 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:33:07.518072 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8571a881-ec9e-4fd1-88b7-b0cc8e443545 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:33:07.522712 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb32a2a-7340-4e27-9d94-04cd32e18d9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:33:07.523314 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb32a2a-7340-4e27-9d94-04cd32e18d9f admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:33:07.523595 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb32a2a-7340-4e27-9d94-04cd32e18d9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:33:07.523869 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb32a2a-7340-4e27-9d94-04cd32e18d9f admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:33:07.524512 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb32a2a-7340-4e27-9d94-04cd32e18d9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:33:07.526181 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb32a2a-7340-4e27-9d94-04cd32e18d9f admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:33:07.537008 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1194/2362] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:33:07 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:33:07.541146 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d60a7f-7cda-4b0f-996d-20567499fee1 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:33:07.550846 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d60a7f-7cda-4b0f-996d-20567499fee1 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:33:07.555474 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128395e3-7281-44b2-87f5-1dd4fbdc9640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:33:07.556118 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128395e3-7281-44b2-87f5-1dd4fbdc9640 admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:33:07.556991 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128395e3-7281-44b2-87f5-1dd4fbdc9640 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:33:07.557213 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128395e3-7281-44b2-87f5-1dd4fbdc9640 admin admin] PATH_INFO: `/v3/projects/04387c5c0c64453c8699eae337b95497/users/88644e22909a4ac084845bc869afa64f/roles/b75cb360925e4ba5a65f14965c103ffd` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:33:07.572810 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128395e3-7281-44b2-87f5-1dd4fbdc9640 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04387c5c0c64453c8699eae337b95497, user_id=88644e22909a4ac084845bc869afa64f, role_id=b75cb360925e4ba5a65f14965c103ffd)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:33:07.575546 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128395e3-7281-44b2-87f5-1dd4fbdc9640 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:33:07.589415 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1169/2363] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:33:07 2022] PUT /identity/v3/projects/04387c5c0c64453c8699eae337b95497/users/88644e22909a4ac084845bc869afa64f/roles/b75cb360925e4ba5a65f14965c103ffd => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:33:07.600006 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1935c576-6391-4221-8063-c9e24edc4b5e None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:33:07.600293 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1935c576-6391-4221-8063-c9e24edc4b5e None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:33:07.600564 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1935c576-6391-4221-8063-c9e24edc4b5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:33:07.628364 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-1935c576-6391-4221-8063-c9e24edc4b5e None None] MFA Rules not processed for user `88644e22909a4ac084845bc869afa64f`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:33:07.629607 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-1935c576-6391-4221-8063-c9e24edc4b5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:33:07.695843 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1195/2364] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:33:07 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:33:07.707611 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128395e3-7281-44b2-87f5-1dd4fbdc9640 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:33:07.709277 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-128395e3-7281-44b2-87f5-1dd4fbdc9640 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:33:07.723350 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:33:07.771009 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128395e3-7281-44b2-87f5-1dd4fbdc9640 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:33:07.774903 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09f50b3-1638-4039-93e0-c895270c669c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '523ed589872a45a09b6a7675a7fb0b75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:33:07.776131 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09f50b3-1638-4039-93e0-c895270c669c service cinder] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:33:07.776398 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09f50b3-1638-4039-93e0-c895270c669c service cinder] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:33:07.776665 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09f50b3-1638-4039-93e0-c895270c669c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:33:07.784751 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09f50b3-1638-4039-93e0-c895270c669c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:33:07.786283 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09f50b3-1638-4039-93e0-c895270c669c service cinder] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:33:07.795308 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1170/2365] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:33:07 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:33:08.075643 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1935c576-6391-4221-8063-c9e24edc4b5e None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:33:08.077689 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-1935c576-6391-4221-8063-c9e24edc4b5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:33:08.151453 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1935c576-6391-4221-8063-c9e24edc4b5e None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:33:08.156168 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f34ea6d-50a8-48f7-9467-49e47062e4d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:33:08.156802 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f34ea6d-50a8-48f7-9467-49e47062e4d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:33:08.157088 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f34ea6d-50a8-48f7-9467-49e47062e4d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:33:08.157391 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f34ea6d-50a8-48f7-9467-49e47062e4d8 admin admin] PATH_INFO: `/v3/users/574437ec44d64cbe86ca5e61f3ec4f1a` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:33:08.158518 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f34ea6d-50a8-48f7-9467-49e47062e4d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=574437ec44d64cbe86ca5e61f3ec4f1a)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:33:08.160453 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f34ea6d-50a8-48f7-9467-49e47062e4d8 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:33:08.208374 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-1f34ea6d-50a8-48f7-9467-49e47062e4d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '574437ec44d64cbe86ca5e61f3ec4f1a'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:33:08.219313 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:33:08.226737 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-1f34ea6d-50a8-48f7-9467-49e47062e4d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '574437ec44d64cbe86ca5e61f3ec4f1a'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:33:08.238335 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-1f34ea6d-50a8-48f7-9467-49e47062e4d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '574437ec44d64cbe86ca5e61f3ec4f1a'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:33:08.239095 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-1f34ea6d-50a8-48f7-9467-49e47062e4d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '574437ec44d64cbe86ca5e61f3ec4f1a'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:33:08.239793 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-1f34ea6d-50a8-48f7-9467-49e47062e4d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '574437ec44d64cbe86ca5e61f3ec4f1a'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:33:08.259051 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1196/2366] 10.222.0.102 () {58 vars in 1229 bytes} [Thu Sep 1 17:33:08 2022] DELETE /identity/v3/users/574437ec44d64cbe86ca5e61f3ec4f1a => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:33:08.263783 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3399dbee-f04d-4340-bd6d-1bafa93a3e35 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:33:08.264247 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3399dbee-f04d-4340-bd6d-1bafa93a3e35 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:33:08.264367 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3399dbee-f04d-4340-bd6d-1bafa93a3e35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:33:08.288924 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-3399dbee-f04d-4340-bd6d-1bafa93a3e35 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:33:08.290281 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-3399dbee-f04d-4340-bd6d-1bafa93a3e35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:33:08.363290 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1171/2367] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:33:08 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:33:08.373502 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f34ea6d-50a8-48f7-9467-49e47062e4d8 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:33:08.374952 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-1f34ea6d-50a8-48f7-9467-49e47062e4d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:33:08.440748 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f34ea6d-50a8-48f7-9467-49e47062e4d8 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:33:08.444908 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516065fa-bbac-426d-bf52-ebe5cd5a1070 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:33:08.445570 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516065fa-bbac-426d-bf52-ebe5cd5a1070 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:33:08.445955 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516065fa-bbac-426d-bf52-ebe5cd5a1070 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:33:08.446247 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516065fa-bbac-426d-bf52-ebe5cd5a1070 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:33:08.456379 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516065fa-bbac-426d-bf52-ebe5cd5a1070 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:33:08.457869 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516065fa-bbac-426d-bf52-ebe5cd5a1070 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:33:08.467971 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1197/2368] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:33:08 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:33:08.782976 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3399dbee-f04d-4340-bd6d-1bafa93a3e35 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:33:08.784499 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-3399dbee-f04d-4340-bd6d-1bafa93a3e35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:33:08.812560 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3399dbee-f04d-4340-bd6d-1bafa93a3e35 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:33:08.817446 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcf3918-c2cd-4954-8d6f-b3819b3389e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:33:08.818554 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcf3918-c2cd-4954-8d6f-b3819b3389e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:33:08.819233 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcf3918-c2cd-4954-8d6f-b3819b3389e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:33:08.819728 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcf3918-c2cd-4954-8d6f-b3819b3389e0 admin admin] PATH_INFO: `/v3/projects/1c5b780c53444adf9c22202c0aa3d339` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:33:08.823152 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcf3918-c2cd-4954-8d6f-b3819b3389e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c5b780c53444adf9c22202c0aa3d339)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:33:08.825870 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcf3918-c2cd-4954-8d6f-b3819b3389e0 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:33:08.884287 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-1bcf3918-c2cd-4954-8d6f-b3819b3389e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c5b780c53444adf9c22202c0aa3d339', 'request_id': 'req-1bcf3918-c2cd-4954-8d6f-b3819b3389e0'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:33:08.893584 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-1bcf3918-c2cd-4954-8d6f-b3819b3389e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c5b780c53444adf9c22202c0aa3d339', 'request_id': 'req-1bcf3918-c2cd-4954-8d6f-b3819b3389e0'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:33:08.904234 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-1bcf3918-c2cd-4954-8d6f-b3819b3389e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c5b780c53444adf9c22202c0aa3d339', 'request_id': 'req-1bcf3918-c2cd-4954-8d6f-b3819b3389e0'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:33:08.919348 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-1bcf3918-c2cd-4954-8d6f-b3819b3389e0 admin admin] The token cache is being invalidate because project 1c5b780c53444adf9c22202c0aa3d339 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:33:08.919837 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-1bcf3918-c2cd-4954-8d6f-b3819b3389e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:33:08.920692 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-1bcf3918-c2cd-4954-8d6f-b3819b3389e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:33:08.922322 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1172/2369] 10.222.0.102 () {58 vars in 1238 bytes} [Thu Sep 1 17:33:08 2022] DELETE /identity/v3/projects/1c5b780c53444adf9c22202c0aa3d339 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:33:08.927178 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107cd37f-c712-4141-8fea-d78a8302ff75 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:33:08.927464 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107cd37f-c712-4141-8fea-d78a8302ff75 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:33:08.927741 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107cd37f-c712-4141-8fea-d78a8302ff75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:33:08.946860 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-107cd37f-c712-4141-8fea-d78a8302ff75 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:33:08.948085 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-107cd37f-c712-4141-8fea-d78a8302ff75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:33:09.015523 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1198/2370] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:33:08 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:33:09.023443 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fc84d2-5d49-450b-a2e1-c3cf244b86c2 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:33:09.023792 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fc84d2-5d49-450b-a2e1-c3cf244b86c2 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:33:09.024072 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fc84d2-5d49-450b-a2e1-c3cf244b86c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:33:09.045634 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-f6fc84d2-5d49-450b-a2e1-c3cf244b86c2 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:33:09.046878 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-f6fc84d2-5d49-450b-a2e1-c3cf244b86c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:33:09.055098 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1173/2371] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:33:09 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:33:09.059417 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107cd37f-c712-4141-8fea-d78a8302ff75 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:33:09.070700 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107cd37f-c712-4141-8fea-d78a8302ff75 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:33:09.075297 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c67909-39d7-4dda-9ca4-530219cc8232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:33:09.075892 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c67909-39d7-4dda-9ca4-530219cc8232 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:33:09.076171 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c67909-39d7-4dda-9ca4-530219cc8232 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:33:09.076443 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c67909-39d7-4dda-9ca4-530219cc8232 admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:33:09.077088 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c67909-39d7-4dda-9ca4-530219cc8232 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:33:09.077436 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c67909-39d7-4dda-9ca4-530219cc8232 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:33:09.079154 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c67909-39d7-4dda-9ca4-530219cc8232 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:33:09.092710 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1199/2372] 10.222.0.102 () {58 vars in 1158 bytes} [Thu Sep 1 17:33:09 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:33:09.097237 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fc84d2-5d49-450b-a2e1-c3cf244b86c2 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:33:09.107479 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fc84d2-5d49-450b-a2e1-c3cf244b86c2 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:33:09.112158 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae65f6e0-2bf7-4cff-9484-b67033ee04fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:33:09.112753 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae65f6e0-2bf7-4cff-9484-b67033ee04fd admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:33:09.113032 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae65f6e0-2bf7-4cff-9484-b67033ee04fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:33:09.113323 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae65f6e0-2bf7-4cff-9484-b67033ee04fd admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:33:09.113963 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae65f6e0-2bf7-4cff-9484-b67033ee04fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:33:09.115859 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae65f6e0-2bf7-4cff-9484-b67033ee04fd admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:33:09.149359 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1174/2373] 10.222.0.102 () {60 vars in 1158 bytes} [Thu Sep 1 17:33:09 2022] POST /identity/v3/projects => generated 398 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:33:09.151404 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c67909-39d7-4dda-9ca4-530219cc8232 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:33:09.162500 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c67909-39d7-4dda-9ca4-530219cc8232 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:33:09.237437 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655e382e-73ed-48b4-b68c-a57884ea1f41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:33:09.237971 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655e382e-73ed-48b4-b68c-a57884ea1f41 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:33:09.238256 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655e382e-73ed-48b4-b68c-a57884ea1f41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:33:09.238531 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655e382e-73ed-48b4-b68c-a57884ea1f41 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:33:09.239163 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655e382e-73ed-48b4-b68c-a57884ea1f41 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:33:09.241125 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655e382e-73ed-48b4-b68c-a57884ea1f41 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:33:09.253244 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:33:09.266211 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1200/2374] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:33:09 2022] POST /identity/v3/users => generated 300 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:33:09.270471 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae65f6e0-2bf7-4cff-9484-b67033ee04fd admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:33:09.282055 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae65f6e0-2bf7-4cff-9484-b67033ee04fd admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:33:09.286624 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2061a84b-14fc-4d4c-87d9-8853289bcb61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:33:09.287211 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2061a84b-14fc-4d4c-87d9-8853289bcb61 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:33:09.287487 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2061a84b-14fc-4d4c-87d9-8853289bcb61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:33:09.287760 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2061a84b-14fc-4d4c-87d9-8853289bcb61 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:33:09.288387 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2061a84b-14fc-4d4c-87d9-8853289bcb61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:33:09.290018 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2061a84b-14fc-4d4c-87d9-8853289bcb61 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:33:09.292008 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:33:09.301753 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1175/2375] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:33:09 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:33:09.306847 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655e382e-73ed-48b4-b68c-a57884ea1f41 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:33:09.316649 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655e382e-73ed-48b4-b68c-a57884ea1f41 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:33:09.321162 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c48ce34-4df7-4c6e-aa31-125f518ae0e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:33:09.321781 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c48ce34-4df7-4c6e-aa31-125f518ae0e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:33:09.322551 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c48ce34-4df7-4c6e-aa31-125f518ae0e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:33:09.322818 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c48ce34-4df7-4c6e-aa31-125f518ae0e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:33:09.323446 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c48ce34-4df7-4c6e-aa31-125f518ae0e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:33:09.325024 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c48ce34-4df7-4c6e-aa31-125f518ae0e6 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:33:09.335376 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1201/2376] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:33:09 2022] GET /identity/v3/roles => generated 1427 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:33:09.340035 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2061a84b-14fc-4d4c-87d9-8853289bcb61 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:33:09.349999 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2061a84b-14fc-4d4c-87d9-8853289bcb61 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:33:09.354564 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a738a7c1-3295-40db-874c-e35ab0190480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:33:09.355204 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a738a7c1-3295-40db-874c-e35ab0190480 admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:33:09.355486 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a738a7c1-3295-40db-874c-e35ab0190480 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:33:09.355754 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a738a7c1-3295-40db-874c-e35ab0190480 admin admin] PATH_INFO: `/v3/projects/5c686f06e4a74c4a87a29f86a53653ed/users/6827453b6aa9434389bfc8322cd312cd/roles/b75cb360925e4ba5a65f14965c103ffd` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:33:09.370030 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a738a7c1-3295-40db-874c-e35ab0190480 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c686f06e4a74c4a87a29f86a53653ed, user_id=6827453b6aa9434389bfc8322cd312cd, role_id=b75cb360925e4ba5a65f14965c103ffd)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:33:09.372652 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a738a7c1-3295-40db-874c-e35ab0190480 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:33:09.385403 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1176/2377] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:33:09 2022] PUT /identity/v3/projects/5c686f06e4a74c4a87a29f86a53653ed/users/6827453b6aa9434389bfc8322cd312cd/roles/b75cb360925e4ba5a65f14965c103ffd => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:33:09.395182 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e854d8b-7713-4009-b61a-8c0c736a1c5b None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:33:09.395487 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e854d8b-7713-4009-b61a-8c0c736a1c5b None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:33:09.395759 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e854d8b-7713-4009-b61a-8c0c736a1c5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:33:09.423651 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-3e854d8b-7713-4009-b61a-8c0c736a1c5b None None] MFA Rules not processed for user `6827453b6aa9434389bfc8322cd312cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:33:09.424852 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-3e854d8b-7713-4009-b61a-8c0c736a1c5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:33:09.491232 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1202/2378] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:33:09 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:33:09.502431 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a738a7c1-3295-40db-874c-e35ab0190480 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:33:09.504353 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-a738a7c1-3295-40db-874c-e35ab0190480 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:33:09.570365 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a738a7c1-3295-40db-874c-e35ab0190480 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:33:09.575995 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13edf6a9-56f9-462a-bfba-704803ee2aa6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '523ed589872a45a09b6a7675a7fb0b75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:33:09.576609 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13edf6a9-56f9-462a-bfba-704803ee2aa6 service cinder] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:33:09.576887 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13edf6a9-56f9-462a-bfba-704803ee2aa6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:33:09.577159 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13edf6a9-56f9-462a-bfba-704803ee2aa6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:33:09.585891 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13edf6a9-56f9-462a-bfba-704803ee2aa6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:33:09.587314 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13edf6a9-56f9-462a-bfba-704803ee2aa6 service cinder] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:33:09.596623 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1177/2379] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:33:09 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:33:19.488186 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=39 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:33:19.592454 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=37,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:33:30.036796 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e854d8b-7713-4009-b61a-8c0c736a1c5b None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:33:30.097990 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e854d8b-7713-4009-b61a-8c0c736a1c5b None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:33:30.102562 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0408b869-a5ab-4259-b32d-04b13b2965a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:33:30.103219 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0408b869-a5ab-4259-b32d-04b13b2965a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:33:30.103502 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0408b869-a5ab-4259-b32d-04b13b2965a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:33:30.103777 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0408b869-a5ab-4259-b32d-04b13b2965a7 admin admin] PATH_INFO: `/v3/users/6827453b6aa9434389bfc8322cd312cd` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:33:30.104907 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0408b869-a5ab-4259-b32d-04b13b2965a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6827453b6aa9434389bfc8322cd312cd)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:33:30.106870 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0408b869-a5ab-4259-b32d-04b13b2965a7 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:33:30.153655 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-0408b869-a5ab-4259-b32d-04b13b2965a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6827453b6aa9434389bfc8322cd312cd'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:33:30.165976 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-0408b869-a5ab-4259-b32d-04b13b2965a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6827453b6aa9434389bfc8322cd312cd'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:33:30.173732 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-0408b869-a5ab-4259-b32d-04b13b2965a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6827453b6aa9434389bfc8322cd312cd'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:33:30.174392 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-0408b869-a5ab-4259-b32d-04b13b2965a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6827453b6aa9434389bfc8322cd312cd'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:33:30.175011 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-0408b869-a5ab-4259-b32d-04b13b2965a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6827453b6aa9434389bfc8322cd312cd'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:33:30.188245 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1203/2380] 10.222.0.102 () {58 vars in 1229 bytes} [Thu Sep 1 17:33:30 2022] DELETE /identity/v3/users/6827453b6aa9434389bfc8322cd312cd => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:33:30.192678 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db7bd99-1a74-4681-8f10-41146ffe9fb7 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:33:30.193181 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db7bd99-1a74-4681-8f10-41146ffe9fb7 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:33:30.193388 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db7bd99-1a74-4681-8f10-41146ffe9fb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:33:30.213509 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-7db7bd99-1a74-4681-8f10-41146ffe9fb7 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:33:30.214746 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-7db7bd99-1a74-4681-8f10-41146ffe9fb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:33:30.283282 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1178/2381] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:33:30 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:33:30.292929 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0408b869-a5ab-4259-b32d-04b13b2965a7 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:33:30.294365 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-0408b869-a5ab-4259-b32d-04b13b2965a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:33:30.351527 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0408b869-a5ab-4259-b32d-04b13b2965a7 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:33:30.355104 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87258f5c-a074-4482-a59d-0e8238202827 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:33:30.355706 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87258f5c-a074-4482-a59d-0e8238202827 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:33:30.355978 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87258f5c-a074-4482-a59d-0e8238202827 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:33:30.356247 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87258f5c-a074-4482-a59d-0e8238202827 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:33:30.365095 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87258f5c-a074-4482-a59d-0e8238202827 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:33:30.366579 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87258f5c-a074-4482-a59d-0e8238202827 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:33:30.376820 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1204/2382] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:33:30 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:33:30.683512 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db7bd99-1a74-4681-8f10-41146ffe9fb7 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:33:30.685009 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-7db7bd99-1a74-4681-8f10-41146ffe9fb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:33:30.697060 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db7bd99-1a74-4681-8f10-41146ffe9fb7 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:33:30.701984 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d76d6aa-8673-4daf-8ee2-53b59e05e5a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:33:30.702601 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d76d6aa-8673-4daf-8ee2-53b59e05e5a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:33:30.703260 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d76d6aa-8673-4daf-8ee2-53b59e05e5a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:33:30.703535 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d76d6aa-8673-4daf-8ee2-53b59e05e5a9 admin admin] PATH_INFO: `/v3/projects/5c686f06e4a74c4a87a29f86a53653ed` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:33:30.704667 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d76d6aa-8673-4daf-8ee2-53b59e05e5a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c686f06e4a74c4a87a29f86a53653ed)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:33:30.706585 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d76d6aa-8673-4daf-8ee2-53b59e05e5a9 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:33:30.755652 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-7d76d6aa-8673-4daf-8ee2-53b59e05e5a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c686f06e4a74c4a87a29f86a53653ed', 'request_id': 'req-7d76d6aa-8673-4daf-8ee2-53b59e05e5a9'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:33:30.761238 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-7d76d6aa-8673-4daf-8ee2-53b59e05e5a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c686f06e4a74c4a87a29f86a53653ed', 'request_id': 'req-7d76d6aa-8673-4daf-8ee2-53b59e05e5a9'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:33:30.768514 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-7d76d6aa-8673-4daf-8ee2-53b59e05e5a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c686f06e4a74c4a87a29f86a53653ed', 'request_id': 'req-7d76d6aa-8673-4daf-8ee2-53b59e05e5a9'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:33:30.779560 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-7d76d6aa-8673-4daf-8ee2-53b59e05e5a9 admin admin] The token cache is being invalidate because project 5c686f06e4a74c4a87a29f86a53653ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:33:30.779883 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-7d76d6aa-8673-4daf-8ee2-53b59e05e5a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:33:30.780538 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-7d76d6aa-8673-4daf-8ee2-53b59e05e5a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:33:30.782027 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1179/2383] 10.222.0.102 () {58 vars in 1238 bytes} [Thu Sep 1 17:33:30 2022] DELETE /identity/v3/projects/5c686f06e4a74c4a87a29f86a53653ed => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:33:40.372636 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:33:40.545743 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87258f5c-a074-4482-a59d-0e8238202827 service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:33:40.547352 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-87258f5c-a074-4482-a59d-0e8238202827 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:33:40.559918 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87258f5c-a074-4482-a59d-0e8238202827 service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:33:40.616700 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f9545a-a62f-4acb-8cdc-6d597213a122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:33:40.617241 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f9545a-a62f-4acb-8cdc-6d597213a122 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:33:40.617562 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f9545a-a62f-4acb-8cdc-6d597213a122 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:33:40.617853 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f9545a-a62f-4acb-8cdc-6d597213a122 admin admin] PATH_INFO: `/v3/users/88644e22909a4ac084845bc869afa64f` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:33:40.618965 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f9545a-a62f-4acb-8cdc-6d597213a122 admin admin] RBAC: Authorizing `identity:delete_user(user_id=88644e22909a4ac084845bc869afa64f)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:33:40.620879 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f9545a-a62f-4acb-8cdc-6d597213a122 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:33:40.671727 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-89f9545a-a62f-4acb-8cdc-6d597213a122 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88644e22909a4ac084845bc869afa64f'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:33:40.683937 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-89f9545a-a62f-4acb-8cdc-6d597213a122 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88644e22909a4ac084845bc869afa64f'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:33:40.692929 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-89f9545a-a62f-4acb-8cdc-6d597213a122 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88644e22909a4ac084845bc869afa64f'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:33:40.693577 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-89f9545a-a62f-4acb-8cdc-6d597213a122 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88644e22909a4ac084845bc869afa64f'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:33:40.694189 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-89f9545a-a62f-4acb-8cdc-6d597213a122 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88644e22909a4ac084845bc869afa64f'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:33:40.708073 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1205/2384] 10.222.0.102 () {58 vars in 1229 bytes} [Thu Sep 1 17:33:40 2022] DELETE /identity/v3/users/88644e22909a4ac084845bc869afa64f => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:33:40.712916 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415370f5-8dbf-4acd-8599-36f397c4719d None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:33:40.713406 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415370f5-8dbf-4acd-8599-36f397c4719d None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:33:40.713502 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415370f5-8dbf-4acd-8599-36f397c4719d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:33:40.732477 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-415370f5-8dbf-4acd-8599-36f397c4719d None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:33:40.733714 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-415370f5-8dbf-4acd-8599-36f397c4719d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:33:40.793694 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1180/2385] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:33:40 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:33:40.803262 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f9545a-a62f-4acb-8cdc-6d597213a122 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:33:40.804644 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-89f9545a-a62f-4acb-8cdc-6d597213a122 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:33:40.871559 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f9545a-a62f-4acb-8cdc-6d597213a122 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:33:40.875265 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793a67bb-a606-4c0b-a4a4-89b127710fb9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:33:40.875878 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793a67bb-a606-4c0b-a4a4-89b127710fb9 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:33:40.876161 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793a67bb-a606-4c0b-a4a4-89b127710fb9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:33:40.876435 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793a67bb-a606-4c0b-a4a4-89b127710fb9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:33:40.886549 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793a67bb-a606-4c0b-a4a4-89b127710fb9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:33:40.888047 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793a67bb-a606-4c0b-a4a4-89b127710fb9 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:33:40.897623 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1206/2386] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:33:40 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:33:41.272052 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415370f5-8dbf-4acd-8599-36f397c4719d None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:33:41.274016 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-415370f5-8dbf-4acd-8599-36f397c4719d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:33:41.297499 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415370f5-8dbf-4acd-8599-36f397c4719d None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:33:41.302571 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8bfab4-b351-4243-9d22-009b98a3825b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:33:41.303303 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8bfab4-b351-4243-9d22-009b98a3825b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:33:41.303547 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8bfab4-b351-4243-9d22-009b98a3825b admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:33:41.303828 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8bfab4-b351-4243-9d22-009b98a3825b admin admin] PATH_INFO: `/v3/projects/04387c5c0c64453c8699eae337b95497` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:33:41.305065 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8bfab4-b351-4243-9d22-009b98a3825b admin admin] RBAC: Authorizing `identity:delete_project(project_id=04387c5c0c64453c8699eae337b95497)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:33:41.307054 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8bfab4-b351-4243-9d22-009b98a3825b admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:33:41.342476 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:33:41.393239 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-9a8bfab4-b351-4243-9d22-009b98a3825b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04387c5c0c64453c8699eae337b95497', 'request_id': 'req-9a8bfab4-b351-4243-9d22-009b98a3825b'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:33:41.422349 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-9a8bfab4-b351-4243-9d22-009b98a3825b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04387c5c0c64453c8699eae337b95497', 'request_id': 'req-9a8bfab4-b351-4243-9d22-009b98a3825b'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:33:41.443387 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-9a8bfab4-b351-4243-9d22-009b98a3825b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04387c5c0c64453c8699eae337b95497', 'request_id': 'req-9a8bfab4-b351-4243-9d22-009b98a3825b'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:33:41.469912 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-9a8bfab4-b351-4243-9d22-009b98a3825b admin admin] The token cache is being invalidate because project 04387c5c0c64453c8699eae337b95497 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:33:41.471631 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-9a8bfab4-b351-4243-9d22-009b98a3825b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:33:41.474457 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-9a8bfab4-b351-4243-9d22-009b98a3825b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:33:41.482453 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1181/2387] 10.222.0.102 () {58 vars in 1238 bytes} [Thu Sep 1 17:33:41 2022] DELETE /identity/v3/projects/04387c5c0c64453c8699eae337b95497 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:33:41.497604 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ee937d-ea2d-4c5c-8561-e712b4bccfaf None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:33:41.497604 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ee937d-ea2d-4c5c-8561-e712b4bccfaf None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:33:41.497604 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ee937d-ea2d-4c5c-8561-e712b4bccfaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:33:41.520826 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-63ee937d-ea2d-4c5c-8561-e712b4bccfaf None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:33:41.523438 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-63ee937d-ea2d-4c5c-8561-e712b4bccfaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:33:41.559774 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:33:41.642803 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1207/2388] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:33:41 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:33:41.651420 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dfb1a7-839e-4ff2-9887-2bd5aa4247a3 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:33:41.651633 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dfb1a7-839e-4ff2-9887-2bd5aa4247a3 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:33:41.651907 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dfb1a7-839e-4ff2-9887-2bd5aa4247a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:33:41.684087 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-a4dfb1a7-839e-4ff2-9887-2bd5aa4247a3 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:33:41.686345 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-a4dfb1a7-839e-4ff2-9887-2bd5aa4247a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:33:41.697342 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1182/2389] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:33:41 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:33:41.700276 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ee937d-ea2d-4c5c-8561-e712b4bccfaf None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:33:41.712906 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ee937d-ea2d-4c5c-8561-e712b4bccfaf None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:33:41.719704 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe68a11-f406-4d4f-8e8a-fe0984442644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:33:41.719704 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe68a11-f406-4d4f-8e8a-fe0984442644 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:33:41.719704 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe68a11-f406-4d4f-8e8a-fe0984442644 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:33:41.719704 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe68a11-f406-4d4f-8e8a-fe0984442644 admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:33:41.720130 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe68a11-f406-4d4f-8e8a-fe0984442644 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:33:41.720130 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe68a11-f406-4d4f-8e8a-fe0984442644 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:33:41.721887 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe68a11-f406-4d4f-8e8a-fe0984442644 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:33:41.738927 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1208/2390] 10.222.0.102 () {58 vars in 1158 bytes} [Thu Sep 1 17:33:41 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:33:41.739580 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dfb1a7-839e-4ff2-9887-2bd5aa4247a3 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:33:41.752021 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dfb1a7-839e-4ff2-9887-2bd5aa4247a3 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:33:41.757731 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cad5a87-c66c-4220-a448-604bdac8bbb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:33:41.762518 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cad5a87-c66c-4220-a448-604bdac8bbb6 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:33:41.762809 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cad5a87-c66c-4220-a448-604bdac8bbb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:33:41.763088 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cad5a87-c66c-4220-a448-604bdac8bbb6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:33:41.763750 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cad5a87-c66c-4220-a448-604bdac8bbb6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:33:41.767640 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cad5a87-c66c-4220-a448-604bdac8bbb6 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:33:41.814876 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe68a11-f406-4d4f-8e8a-fe0984442644 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:33:41.817651 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1183/2391] 10.222.0.102 () {60 vars in 1158 bytes} [Thu Sep 1 17:33:41 2022] POST /identity/v3/projects => generated 380 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:33:41.833153 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe68a11-f406-4d4f-8e8a-fe0984442644 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:33:41.942586 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fced213-60b3-4f63-955c-d8a2acd19f94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:33:41.943308 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fced213-60b3-4f63-955c-d8a2acd19f94 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:33:41.943703 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fced213-60b3-4f63-955c-d8a2acd19f94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:33:41.944093 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fced213-60b3-4f63-955c-d8a2acd19f94 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:33:41.944818 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fced213-60b3-4f63-955c-d8a2acd19f94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:33:41.947931 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fced213-60b3-4f63-955c-d8a2acd19f94 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:33:42.004458 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1209/2392] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:33:41 2022] POST /identity/v3/users => generated 291 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:33:42.009134 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cad5a87-c66c-4220-a448-604bdac8bbb6 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:33:42.025872 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cad5a87-c66c-4220-a448-604bdac8bbb6 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:33:42.030790 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536169b6-24c8-4a7f-a87d-f803938b6e37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:33:42.032110 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536169b6-24c8-4a7f-a87d-f803938b6e37 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:33:42.032396 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536169b6-24c8-4a7f-a87d-f803938b6e37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:33:42.034444 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536169b6-24c8-4a7f-a87d-f803938b6e37 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:33:42.034444 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536169b6-24c8-4a7f-a87d-f803938b6e37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:33:42.034946 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536169b6-24c8-4a7f-a87d-f803938b6e37 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:33:42.059727 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1184/2393] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:33:42 2022] GET /identity/v3/roles => generated 1427 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:33:42.063475 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fced213-60b3-4f63-955c-d8a2acd19f94 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:33:42.074065 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fced213-60b3-4f63-955c-d8a2acd19f94 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:33:42.079544 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f8e3ae-339d-4d5c-a073-bc0be3d574bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:33:42.080297 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f8e3ae-339d-4d5c-a073-bc0be3d574bf admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:33:42.080641 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f8e3ae-339d-4d5c-a073-bc0be3d574bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:33:42.080915 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f8e3ae-339d-4d5c-a073-bc0be3d574bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:33:42.081716 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f8e3ae-339d-4d5c-a073-bc0be3d574bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:33:42.083474 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f8e3ae-339d-4d5c-a073-bc0be3d574bf admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:33:42.101690 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536169b6-24c8-4a7f-a87d-f803938b6e37 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:33:42.109059 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1210/2394] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:33:42 2022] GET /identity/v3/roles => generated 1427 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:33:42.114969 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536169b6-24c8-4a7f-a87d-f803938b6e37 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:33:42.120344 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cf7e79-fb47-4ea8-87e6-32c26800130e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:33:42.120814 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cf7e79-fb47-4ea8-87e6-32c26800130e admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:33:42.121247 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cf7e79-fb47-4ea8-87e6-32c26800130e admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:33:42.121557 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cf7e79-fb47-4ea8-87e6-32c26800130e admin admin] PATH_INFO: `/v3/projects/2cc300abf09e4bb3857336df552286e2/users/8f59383e2f6c4da8b683d718f62d25fb/roles/b75cb360925e4ba5a65f14965c103ffd` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:33:42.141605 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cf7e79-fb47-4ea8-87e6-32c26800130e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2cc300abf09e4bb3857336df552286e2, user_id=8f59383e2f6c4da8b683d718f62d25fb, role_id=b75cb360925e4ba5a65f14965c103ffd)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:33:42.144585 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cf7e79-fb47-4ea8-87e6-32c26800130e admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:33:42.165616 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1185/2395] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:33:42 2022] PUT /identity/v3/projects/2cc300abf09e4bb3857336df552286e2/users/8f59383e2f6c4da8b683d718f62d25fb/roles/b75cb360925e4ba5a65f14965c103ffd => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:33:42.169330 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338d7f88-6c5b-4746-8c5c-db4c1c82587e None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:33:42.169637 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338d7f88-6c5b-4746-8c5c-db4c1c82587e None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:33:42.169916 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338d7f88-6c5b-4746-8c5c-db4c1c82587e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:33:42.206674 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-338d7f88-6c5b-4746-8c5c-db4c1c82587e None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:33:42.208209 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-338d7f88-6c5b-4746-8c5c-db4c1c82587e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:33:42.286215 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1211/2396] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:33:42 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:33:42.296175 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cf7e79-fb47-4ea8-87e6-32c26800130e admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:33:42.297881 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-91cf7e79-fb47-4ea8-87e6-32c26800130e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:33:42.378305 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cf7e79-fb47-4ea8-87e6-32c26800130e admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:33:42.382307 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bc6a77-a164-44bb-90a4-d89eaa8a8041 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:33:42.382921 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bc6a77-a164-44bb-90a4-d89eaa8a8041 service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:33:42.383199 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bc6a77-a164-44bb-90a4-d89eaa8a8041 service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:33:42.384135 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bc6a77-a164-44bb-90a4-d89eaa8a8041 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:33:42.393939 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bc6a77-a164-44bb-90a4-d89eaa8a8041 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:33:42.395360 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bc6a77-a164-44bb-90a4-d89eaa8a8041 service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:33:42.405233 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1186/2397] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:33:42 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:33:46.612104 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53018b6c-2dbb-4d5b-bb3e-98a3e5247f30 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:33:46.613049 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53018b6c-2dbb-4d5b-bb3e-98a3e5247f30 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:33:46.613678 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53018b6c-2dbb-4d5b-bb3e-98a3e5247f30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:33:46.651861 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-53018b6c-2dbb-4d5b-bb3e-98a3e5247f30 None None] MFA Rules not processed for user `8f59383e2f6c4da8b683d718f62d25fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:33:46.653087 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-53018b6c-2dbb-4d5b-bb3e-98a3e5247f30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:33:46.671434 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:33:46.727711 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1212/2398] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:33:46 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:33:46.731996 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bc6a77-a164-44bb-90a4-d89eaa8a8041 service neutron] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:33:46.742011 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bc6a77-a164-44bb-90a4-d89eaa8a8041 service neutron] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:33:46.747263 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3069d7-f436-4e91-b440-6cee4d04b328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:33:46.747896 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3069d7-f436-4e91-b440-6cee4d04b328 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:33:46.748174 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3069d7-f436-4e91-b440-6cee4d04b328 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:33:46.748446 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3069d7-f436-4e91-b440-6cee4d04b328 admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:33:46.749072 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3069d7-f436-4e91-b440-6cee4d04b328 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:33:46.751130 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3069d7-f436-4e91-b440-6cee4d04b328 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:33:46.779678 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1187/2399] 10.222.0.102 () {60 vars in 1158 bytes} [Thu Sep 1 17:33:46 2022] POST /identity/v3/projects => generated 378 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:33:46.783592 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53018b6c-2dbb-4d5b-bb3e-98a3e5247f30 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:33:46.794806 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53018b6c-2dbb-4d5b-bb3e-98a3e5247f30 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:33:46.860719 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cff72f-a0dc-40db-bd2f-7b8a4d515248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:33:46.861368 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cff72f-a0dc-40db-bd2f-7b8a4d515248 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:33:46.861714 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cff72f-a0dc-40db-bd2f-7b8a4d515248 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:33:46.861993 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cff72f-a0dc-40db-bd2f-7b8a4d515248 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:33:46.862802 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cff72f-a0dc-40db-bd2f-7b8a4d515248 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:33:46.864743 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cff72f-a0dc-40db-bd2f-7b8a4d515248 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:33:46.890697 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1213/2400] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:33:46 2022] POST /identity/v3/users => generated 296 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:33:46.891809 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3069d7-f436-4e91-b440-6cee4d04b328 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:33:46.902969 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3069d7-f436-4e91-b440-6cee4d04b328 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:33:46.907881 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25d65a9-bc8f-4bca-aaa2-401d460c7463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:33:46.908489 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25d65a9-bc8f-4bca-aaa2-401d460c7463 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:33:46.908795 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25d65a9-bc8f-4bca-aaa2-401d460c7463 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:33:46.909064 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25d65a9-bc8f-4bca-aaa2-401d460c7463 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:33:46.909758 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25d65a9-bc8f-4bca-aaa2-401d460c7463 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:33:46.911373 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25d65a9-bc8f-4bca-aaa2-401d460c7463 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:33:46.922883 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1188/2401] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:33:46 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:33:46.927109 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cff72f-a0dc-40db-bd2f-7b8a4d515248 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:33:46.937879 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cff72f-a0dc-40db-bd2f-7b8a4d515248 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:33:46.942744 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2878113-cea7-48f1-8872-de36cc68ae38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:33:46.943167 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2878113-cea7-48f1-8872-de36cc68ae38 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:33:46.943500 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2878113-cea7-48f1-8872-de36cc68ae38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:33:46.943705 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2878113-cea7-48f1-8872-de36cc68ae38 admin admin] PATH_INFO: `/v3/projects/3833a70bee9f499d82a015a856af5107/users/04a3973bdbb3406f90d28b3f87c3b577/roles/36577a6f46d642d4b81805ede9872490` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:33:46.960568 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2878113-cea7-48f1-8872-de36cc68ae38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3833a70bee9f499d82a015a856af5107, user_id=04a3973bdbb3406f90d28b3f87c3b577, role_id=36577a6f46d642d4b81805ede9872490)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:33:46.963660 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2878113-cea7-48f1-8872-de36cc68ae38 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:33:46.978027 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1214/2402] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:33:46 2022] PUT /identity/v3/projects/3833a70bee9f499d82a015a856af5107/users/04a3973bdbb3406f90d28b3f87c3b577/roles/36577a6f46d642d4b81805ede9872490 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:33:46.984961 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25d65a9-bc8f-4bca-aaa2-401d460c7463 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:33:47.007100 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25d65a9-bc8f-4bca-aaa2-401d460c7463 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:33:47.018785 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:33:47.075024 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dee612-65d9-49bb-bc2b-b9c4764c4eeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:33:47.075638 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dee612-65d9-49bb-bc2b-b9c4764c4eeb admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:33:47.075920 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dee612-65d9-49bb-bc2b-b9c4764c4eeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:33:47.076196 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dee612-65d9-49bb-bc2b-b9c4764c4eeb admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:33:47.076841 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dee612-65d9-49bb-bc2b-b9c4764c4eeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:33:47.078498 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dee612-65d9-49bb-bc2b-b9c4764c4eeb admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:33:47.088920 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1189/2403] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:33:46 2022] GET /identity/v3/roles => generated 1427 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 17:33:47.093476 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2878113-cea7-48f1-8872-de36cc68ae38 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:33:47.104751 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2878113-cea7-48f1-8872-de36cc68ae38 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:33:47.109956 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51238ea5-75b3-4ae1-819f-79ba3bbba4a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:33:47.110704 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51238ea5-75b3-4ae1-819f-79ba3bbba4a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:33:47.110988 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51238ea5-75b3-4ae1-819f-79ba3bbba4a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:33:47.111273 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51238ea5-75b3-4ae1-819f-79ba3bbba4a8 admin admin] PATH_INFO: `/v3/projects/3833a70bee9f499d82a015a856af5107/users/04a3973bdbb3406f90d28b3f87c3b577/roles/36577a6f46d642d4b81805ede9872490` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:33:47.113246 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51238ea5-75b3-4ae1-819f-79ba3bbba4a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3833a70bee9f499d82a015a856af5107, user_id=04a3973bdbb3406f90d28b3f87c3b577, role_id=36577a6f46d642d4b81805ede9872490)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:33:47.115985 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51238ea5-75b3-4ae1-819f-79ba3bbba4a8 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:33:47.127835 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1215/2404] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:33:47 2022] PUT /identity/v3/projects/3833a70bee9f499d82a015a856af5107/users/04a3973bdbb3406f90d28b3f87c3b577/roles/36577a6f46d642d4b81805ede9872490 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:33:50.914077 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c30bc3-d5b8-4d56-ac32-d85df25d6900 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:33:50.914639 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c30bc3-d5b8-4d56-ac32-d85df25d6900 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:33:50.915605 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c30bc3-d5b8-4d56-ac32-d85df25d6900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:33:50.940886 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-95c30bc3-d5b8-4d56-ac32-d85df25d6900 None None] MFA Rules not processed for user `575a76b99ea945efb1024fbb3dc81aaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:33:50.942291 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-95c30bc3-d5b8-4d56-ac32-d85df25d6900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:33:51.031961 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1190/2405] 10.222.0.102 () {58 vars in 1014 bytes} [Thu Sep 1 17:33:50 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:33:51.087731 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51238ea5-75b3-4ae1-819f-79ba3bbba4a8 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:33:51.091195 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-51238ea5-75b3-4ae1-819f-79ba3bbba4a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:33:51.182550 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51238ea5-75b3-4ae1-819f-79ba3bbba4a8 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:33:51.189562 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b300337-1a11-4f32-bf22-783e4db246dc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd55edf6cd1804006aa8269b4c6f80aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:33:51.190423 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b300337-1a11-4f32-bf22-783e4db246dc service swift] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:33:51.190555 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b300337-1a11-4f32-bf22-783e4db246dc service swift] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:33:51.190874 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b300337-1a11-4f32-bf22-783e4db246dc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:33:51.200362 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b300337-1a11-4f32-bf22-783e4db246dc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:33:51.202004 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b300337-1a11-4f32-bf22-783e4db246dc service swift] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:33:51.208127 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:33:51.235961 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1216/2406] 10.222.0.102 () {60 vars in 1389 bytes} [Thu Sep 1 17:33:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 17:33:51.856804 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8853eb9-a27e-4b93-8ae0-6338803eb37b None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:33:51.857265 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8853eb9-a27e-4b93-8ae0-6338803eb37b None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:33:51.857611 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8853eb9-a27e-4b93-8ae0-6338803eb37b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:33:51.926155 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-b8853eb9-a27e-4b93-8ae0-6338803eb37b None None] MFA Rules not processed for user `04a3973bdbb3406f90d28b3f87c3b577`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:33:51.927631 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-b8853eb9-a27e-4b93-8ae0-6338803eb37b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:33:52.027561 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1191/2407] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:33:51 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:33:52.044797 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b300337-1a11-4f32-bf22-783e4db246dc service swift] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:33:52.046918 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-5b300337-1a11-4f32-bf22-783e4db246dc service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:33:52.142021 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b300337-1a11-4f32-bf22-783e4db246dc service swift] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:33:52.149041 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e88eff-cfc1-4387-926c-e7f4faf8d603 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f8293316baf4774b62fc11d55d50865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:33:52.149714 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e88eff-cfc1-4387-926c-e7f4faf8d603 service nova] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:33:52.150011 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e88eff-cfc1-4387-926c-e7f4faf8d603 service nova] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:33:52.150480 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e88eff-cfc1-4387-926c-e7f4faf8d603 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:33:52.233365 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e88eff-cfc1-4387-926c-e7f4faf8d603 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:33:52.235303 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e88eff-cfc1-4387-926c-e7f4faf8d603 service nova] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:33:52.247398 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1217/2408] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:33:52 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:34:02.020465 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=82 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:34:02.241354 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=56 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:34:08.698806 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88802a10-4e43-4ab2-ad2c-909c6a84cfe0 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:34:08.699370 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88802a10-4e43-4ab2-ad2c-909c6a84cfe0 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:34:08.699370 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88802a10-4e43-4ab2-ad2c-909c6a84cfe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:34:08.719615 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-88802a10-4e43-4ab2-ad2c-909c6a84cfe0 None None] MFA Rules not processed for user `d364c74791fc4cd4af6773433a7a13cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:34:08.720962 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-88802a10-4e43-4ab2-ad2c-909c6a84cfe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:34:08.779931 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1192/2409] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:34:08 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:34:18.119798 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e88eff-cfc1-4387-926c-e7f4faf8d603 service nova] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:34:18.121285 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-f7e88eff-cfc1-4387-926c-e7f4faf8d603 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:34:18.190134 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e88eff-cfc1-4387-926c-e7f4faf8d603 service nova] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:34:18.195144 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9768fff6-9f31-4f63-8fd4-494ba708452a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '134ef6039cef468bae0e3f3be79b8073', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:34:18.196292 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9768fff6-9f31-4f63-8fd4-494ba708452a service placement] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:34:18.196575 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9768fff6-9f31-4f63-8fd4-494ba708452a service placement] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:34:18.196851 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9768fff6-9f31-4f63-8fd4-494ba708452a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:34:18.198563 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-9768fff6-9f31-4f63-8fd4-494ba708452a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:34:18.209707 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9768fff6-9f31-4f63-8fd4-494ba708452a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:34:18.211331 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9768fff6-9f31-4f63-8fd4-494ba708452a service placement] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:34:18.221293 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1218/2410] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:34:18 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:34:18.772257 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=25 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:34:22.107883 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997af375-ee7f-4db5-bcf1-724fd22f4553 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:34:22.108443 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997af375-ee7f-4db5-bcf1-724fd22f4553 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:34:22.108443 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997af375-ee7f-4db5-bcf1-724fd22f4553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:34:22.127442 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-997af375-ee7f-4db5-bcf1-724fd22f4553 None None] MFA Rules not processed for user `575a76b99ea945efb1024fbb3dc81aaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:34:22.128610 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-997af375-ee7f-4db5-bcf1-724fd22f4553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:34:22.136158 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1193/2411] 10.222.0.102 () {58 vars in 972 bytes} [Thu Sep 1 17:34:22 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:34:22.148304 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9768fff6-9f31-4f63-8fd4-494ba708452a service placement] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:34:22.159523 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9768fff6-9f31-4f63-8fd4-494ba708452a service placement] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:34:22.164671 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee76401-e33b-4391-a8af-f9aa1b1fa889 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd55edf6cd1804006aa8269b4c6f80aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:34:22.165282 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee76401-e33b-4391-a8af-f9aa1b1fa889 service swift] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:34:22.165610 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee76401-e33b-4391-a8af-f9aa1b1fa889 service swift] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:34:22.165885 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee76401-e33b-4391-a8af-f9aa1b1fa889 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:34:22.174044 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee76401-e33b-4391-a8af-f9aa1b1fa889 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:34:22.175616 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee76401-e33b-4391-a8af-f9aa1b1fa889 service swift] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:34:22.184080 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1219/2412] 10.222.0.102 () {60 vars in 1389 bytes} [Thu Sep 1 17:34:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 17:34:24.129951 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997af375-ee7f-4db5-bcf1-724fd22f4553 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:34:24.131357 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-997af375-ee7f-4db5-bcf1-724fd22f4553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:34:24.199582 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997af375-ee7f-4db5-bcf1-724fd22f4553 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:34:24.204034 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191ff951-1526-46eb-9a16-a6d230ba57ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:34:24.204653 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191ff951-1526-46eb-9a16-a6d230ba57ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:34:24.204931 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191ff951-1526-46eb-9a16-a6d230ba57ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:34:24.205200 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191ff951-1526-46eb-9a16-a6d230ba57ad admin admin] PATH_INFO: `/v3/users/b54b95db13d84fc4b026fdd119cb7ec5` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:34:24.206338 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191ff951-1526-46eb-9a16-a6d230ba57ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=b54b95db13d84fc4b026fdd119cb7ec5)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:34:24.208242 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191ff951-1526-46eb-9a16-a6d230ba57ad admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:34:24.253831 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-191ff951-1526-46eb-9a16-a6d230ba57ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b54b95db13d84fc4b026fdd119cb7ec5'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:34:24.254476 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-191ff951-1526-46eb-9a16-a6d230ba57ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b54b95db13d84fc4b026fdd119cb7ec5'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:34:24.266541 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-191ff951-1526-46eb-9a16-a6d230ba57ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b54b95db13d84fc4b026fdd119cb7ec5'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:34:24.273820 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-191ff951-1526-46eb-9a16-a6d230ba57ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b54b95db13d84fc4b026fdd119cb7ec5'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:34:24.274443 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-191ff951-1526-46eb-9a16-a6d230ba57ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b54b95db13d84fc4b026fdd119cb7ec5'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:34:24.288426 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1194/2413] 10.222.0.102 () {58 vars in 1229 bytes} [Thu Sep 1 17:34:24 2022] DELETE /identity/v3/users/b54b95db13d84fc4b026fdd119cb7ec5 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:34:24.464062 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee76401-e33b-4391-a8af-f9aa1b1fa889 service swift] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:34:24.479446 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee76401-e33b-4391-a8af-f9aa1b1fa889 service swift] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:34:24.537423 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0ca8eb-3e30-4b7e-93b1-084a0def81ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:34:24.538046 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0ca8eb-3e30-4b7e-93b1-084a0def81ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:34:24.538329 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0ca8eb-3e30-4b7e-93b1-084a0def81ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:34:24.539122 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0ca8eb-3e30-4b7e-93b1-084a0def81ec admin admin] PATH_INFO: `/v3/projects/5f5e7805e6ae4547a767b7b382a78082` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:34:24.540225 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0ca8eb-3e30-4b7e-93b1-084a0def81ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f5e7805e6ae4547a767b7b382a78082)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:34:24.542288 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0ca8eb-3e30-4b7e-93b1-084a0def81ec admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:34:24.587098 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-5b0ca8eb-3e30-4b7e-93b1-084a0def81ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f5e7805e6ae4547a767b7b382a78082', 'request_id': 'req-5b0ca8eb-3e30-4b7e-93b1-084a0def81ec'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:34:24.593918 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-5b0ca8eb-3e30-4b7e-93b1-084a0def81ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f5e7805e6ae4547a767b7b382a78082', 'request_id': 'req-5b0ca8eb-3e30-4b7e-93b1-084a0def81ec'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:34:24.600010 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-5b0ca8eb-3e30-4b7e-93b1-084a0def81ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f5e7805e6ae4547a767b7b382a78082', 'request_id': 'req-5b0ca8eb-3e30-4b7e-93b1-084a0def81ec'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:34:24.610913 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-5b0ca8eb-3e30-4b7e-93b1-084a0def81ec admin admin] The token cache is being invalidate because project 5f5e7805e6ae4547a767b7b382a78082 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:34:24.611141 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-5b0ca8eb-3e30-4b7e-93b1-084a0def81ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:34:24.611761 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-5b0ca8eb-3e30-4b7e-93b1-084a0def81ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:34:24.613281 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1220/2414] 10.222.0.102 () {58 vars in 1238 bytes} [Thu Sep 1 17:34:24 2022] DELETE /identity/v3/projects/5f5e7805e6ae4547a767b7b382a78082 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:34:24.618303 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce102190-30cf-4d85-b7bc-974b02244fe0 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:34:24.618593 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce102190-30cf-4d85-b7bc-974b02244fe0 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:34:24.618891 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce102190-30cf-4d85-b7bc-974b02244fe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:34:24.640495 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-ce102190-30cf-4d85-b7bc-974b02244fe0 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:34:24.641749 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-ce102190-30cf-4d85-b7bc-974b02244fe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:34:24.710535 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1195/2415] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:34:24 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:34:24.718455 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edaa9b1-04ef-4ab1-8e21-bbe7be11bfd7 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:34:24.718748 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edaa9b1-04ef-4ab1-8e21-bbe7be11bfd7 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:34:24.719169 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edaa9b1-04ef-4ab1-8e21-bbe7be11bfd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:34:24.733974 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:34:24.740221 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-4edaa9b1-04ef-4ab1-8e21-bbe7be11bfd7 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:34:24.742076 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-4edaa9b1-04ef-4ab1-8e21-bbe7be11bfd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:34:24.750835 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1221/2416] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:34:24 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:34:24.755271 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce102190-30cf-4d85-b7bc-974b02244fe0 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:34:24.765558 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce102190-30cf-4d85-b7bc-974b02244fe0 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:34:24.770189 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8129a56-6fd2-409d-b07d-2934fabb7444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:34:24.770791 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8129a56-6fd2-409d-b07d-2934fabb7444 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:34:24.771072 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8129a56-6fd2-409d-b07d-2934fabb7444 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:34:24.771344 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8129a56-6fd2-409d-b07d-2934fabb7444 admin admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:34:24.771990 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8129a56-6fd2-409d-b07d-2934fabb7444 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:34:24.772380 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8129a56-6fd2-409d-b07d-2934fabb7444 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:34:24.774612 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8129a56-6fd2-409d-b07d-2934fabb7444 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:34:24.783461 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:34:24.787731 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1196/2417] 10.222.0.102 () {58 vars in 1158 bytes} [Thu Sep 1 17:34:24 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:34:24.791997 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edaa9b1-04ef-4ab1-8e21-bbe7be11bfd7 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:34:24.801265 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edaa9b1-04ef-4ab1-8e21-bbe7be11bfd7 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:34:24.806221 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb32e9a-2a2c-4ea1-8004-cdf4b83ba114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:34:24.806837 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb32e9a-2a2c-4ea1-8004-cdf4b83ba114 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:34:24.807547 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb32e9a-2a2c-4ea1-8004-cdf4b83ba114 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:34:24.807663 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb32e9a-2a2c-4ea1-8004-cdf4b83ba114 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:34:24.813080 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb32e9a-2a2c-4ea1-8004-cdf4b83ba114 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:34:24.813080 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb32e9a-2a2c-4ea1-8004-cdf4b83ba114 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:34:24.839720 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1222/2418] 10.222.0.102 () {60 vars in 1158 bytes} [Thu Sep 1 17:34:24 2022] POST /identity/v3/projects => generated 364 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:34:24.844149 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8129a56-6fd2-409d-b07d-2934fabb7444 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:34:24.854395 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8129a56-6fd2-409d-b07d-2934fabb7444 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:34:24.911196 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8814ec92-0a4c-4305-bbf9-ce503fb7dd85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:34:24.911719 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8814ec92-0a4c-4305-bbf9-ce503fb7dd85 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:34:24.911996 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8814ec92-0a4c-4305-bbf9-ce503fb7dd85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:34:24.912269 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8814ec92-0a4c-4305-bbf9-ce503fb7dd85 admin admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:34:24.912900 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8814ec92-0a4c-4305-bbf9-ce503fb7dd85 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:34:24.914865 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8814ec92-0a4c-4305-bbf9-ce503fb7dd85 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:34:24.935475 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1197/2419] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:34:24 2022] POST /identity/v3/users => generated 283 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:34:24.939572 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb32e9a-2a2c-4ea1-8004-cdf4b83ba114 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:34:24.951606 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb32e9a-2a2c-4ea1-8004-cdf4b83ba114 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:34:24.958383 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9153884-c742-467e-ba29-7e538f8700ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:34:24.959260 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9153884-c742-467e-ba29-7e538f8700ef admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:34:24.959682 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9153884-c742-467e-ba29-7e538f8700ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:34:24.960090 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9153884-c742-467e-ba29-7e538f8700ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:34:24.961153 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9153884-c742-467e-ba29-7e538f8700ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:34:24.963699 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9153884-c742-467e-ba29-7e538f8700ef admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:34:24.977384 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1223/2420] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:34:24 2022] GET /identity/v3/roles => generated 1427 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:34:24.981487 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8814ec92-0a4c-4305-bbf9-ce503fb7dd85 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:34:24.993042 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8814ec92-0a4c-4305-bbf9-ce503fb7dd85 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:34:24.997833 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2f4270-b0c5-4065-b507-f9f2ee6b4802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:34:24.998426 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2f4270-b0c5-4065-b507-f9f2ee6b4802 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:34:24.998708 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2f4270-b0c5-4065-b507-f9f2ee6b4802 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:34:24.998984 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2f4270-b0c5-4065-b507-f9f2ee6b4802 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:34:24.999620 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2f4270-b0c5-4065-b507-f9f2ee6b4802 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:34:25.001203 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2f4270-b0c5-4065-b507-f9f2ee6b4802 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:34:25.015022 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1198/2421] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:34:24 2022] GET /identity/v3/roles => generated 1427 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:34:25.019158 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9153884-c742-467e-ba29-7e538f8700ef admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:34:25.029098 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9153884-c742-467e-ba29-7e538f8700ef admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:34:25.037968 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d04094f-64bd-43c0-a790-88b5e0ac7a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:34:25.038590 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d04094f-64bd-43c0-a790-88b5e0ac7a17 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:34:25.038865 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d04094f-64bd-43c0-a790-88b5e0ac7a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:34:25.039134 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d04094f-64bd-43c0-a790-88b5e0ac7a17 admin admin] PATH_INFO: `/v3/projects/8c65801234b84dcc93d2e78dbc03dbda/users/bee7f5f71dec45059c02db6cd8841203/roles/b75cb360925e4ba5a65f14965c103ffd` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:34:25.053704 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d04094f-64bd-43c0-a790-88b5e0ac7a17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c65801234b84dcc93d2e78dbc03dbda, user_id=bee7f5f71dec45059c02db6cd8841203, role_id=b75cb360925e4ba5a65f14965c103ffd)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:34:25.056853 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d04094f-64bd-43c0-a790-88b5e0ac7a17 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:34:25.070718 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1224/2422] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:34:25 2022] PUT /identity/v3/projects/8c65801234b84dcc93d2e78dbc03dbda/users/bee7f5f71dec45059c02db6cd8841203/roles/b75cb360925e4ba5a65f14965c103ffd => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:34:25.081862 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a754dd-f9cc-42b6-a39c-4b1f54743411 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:34:25.082153 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a754dd-f9cc-42b6-a39c-4b1f54743411 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:34:25.082430 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a754dd-f9cc-42b6-a39c-4b1f54743411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:34:25.109440 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-87a754dd-f9cc-42b6-a39c-4b1f54743411 None None] MFA Rules not processed for user `bee7f5f71dec45059c02db6cd8841203`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:34:25.110654 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-87a754dd-f9cc-42b6-a39c-4b1f54743411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:34:25.172839 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1199/2423] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:34:25 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:34:25.184417 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d04094f-64bd-43c0-a790-88b5e0ac7a17 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:34:25.186121 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-1d04094f-64bd-43c0-a790-88b5e0ac7a17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:34:25.240807 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d04094f-64bd-43c0-a790-88b5e0ac7a17 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:34:25.244850 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f39a61-484b-4660-bcb5-bb90a30bca56 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '523ed589872a45a09b6a7675a7fb0b75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:34:25.245539 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f39a61-484b-4660-bcb5-bb90a30bca56 service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:34:25.245839 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f39a61-484b-4660-bcb5-bb90a30bca56 service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:34:25.246112 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f39a61-484b-4660-bcb5-bb90a30bca56 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:34:25.254057 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f39a61-484b-4660-bcb5-bb90a30bca56 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:34:25.255471 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f39a61-484b-4660-bcb5-bb90a30bca56 service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:34:25.264587 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1225/2424] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:34:25 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:34:33.964701 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d80eba9-b001-4a06-bca3-fc28df69d4e2 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:34:33.965238 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d80eba9-b001-4a06-bca3-fc28df69d4e2 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:34:33.965238 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d80eba9-b001-4a06-bca3-fc28df69d4e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:34:33.984615 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-4d80eba9-b001-4a06-bca3-fc28df69d4e2 None None] MFA Rules not processed for user `8f59383e2f6c4da8b683d718f62d25fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:34:33.985894 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-4d80eba9-b001-4a06-bca3-fc28df69d4e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:34:34.041479 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1200/2425] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:34:33 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:34:35.260670 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=50,keystone:INSERT=2 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:34:44.034707 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=91,keystone:INSERT=3 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:34:58.275686 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f39a61-484b-4660-bcb5-bb90a30bca56 service cinder] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:34:58.277154 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-c2f39a61-484b-4660-bcb5-bb90a30bca56 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:34:58.335982 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f39a61-484b-4660-bcb5-bb90a30bca56 service cinder] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:34:58.339506 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e14ddcd-9318-46d0-bb4d-63fbfce50b03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:34:58.340371 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e14ddcd-9318-46d0-bb4d-63fbfce50b03 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:34:58.340655 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e14ddcd-9318-46d0-bb4d-63fbfce50b03 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:34:58.340926 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e14ddcd-9318-46d0-bb4d-63fbfce50b03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:34:58.343286 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-5e14ddcd-9318-46d0-bb4d-63fbfce50b03 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:34:58.406623 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e14ddcd-9318-46d0-bb4d-63fbfce50b03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:34:58.408062 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e14ddcd-9318-46d0-bb4d-63fbfce50b03 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:34:58.418051 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1226/2426] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:34:58 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:35:08.413810 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=50 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:35:19.547159 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d80eba9-b001-4a06-bca3-fc28df69d4e2 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:35:19.614413 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d80eba9-b001-4a06-bca3-fc28df69d4e2 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:35:19.618810 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5224c7bf-aace-4fba-a17f-ecf749f66657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:35:19.619418 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5224c7bf-aace-4fba-a17f-ecf749f66657 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:35:19.619693 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5224c7bf-aace-4fba-a17f-ecf749f66657 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:35:19.619962 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5224c7bf-aace-4fba-a17f-ecf749f66657 admin admin] PATH_INFO: `/v3/users/bee7f5f71dec45059c02db6cd8841203` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:35:19.621082 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5224c7bf-aace-4fba-a17f-ecf749f66657 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bee7f5f71dec45059c02db6cd8841203)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:35:19.623015 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5224c7bf-aace-4fba-a17f-ecf749f66657 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:35:19.667846 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-5224c7bf-aace-4fba-a17f-ecf749f66657 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bee7f5f71dec45059c02db6cd8841203'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:35:19.668533 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-5224c7bf-aace-4fba-a17f-ecf749f66657 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bee7f5f71dec45059c02db6cd8841203'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:35:19.681734 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-5224c7bf-aace-4fba-a17f-ecf749f66657 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bee7f5f71dec45059c02db6cd8841203'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:35:19.689054 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-5224c7bf-aace-4fba-a17f-ecf749f66657 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bee7f5f71dec45059c02db6cd8841203'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:35:19.689719 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-5224c7bf-aace-4fba-a17f-ecf749f66657 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bee7f5f71dec45059c02db6cd8841203'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:35:19.702012 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1201/2427] 10.222.0.102 () {58 vars in 1229 bytes} [Thu Sep 1 17:35:19 2022] DELETE /identity/v3/users/bee7f5f71dec45059c02db6cd8841203 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:35:19.706555 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b19839-492c-45e9-9587-94498c57d685 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:35:19.706952 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b19839-492c-45e9-9587-94498c57d685 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:35:19.707139 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b19839-492c-45e9-9587-94498c57d685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:35:19.727951 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-b0b19839-492c-45e9-9587-94498c57d685 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:35:19.729173 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-b0b19839-492c-45e9-9587-94498c57d685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:35:19.790410 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1227/2428] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:35:19 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:35:19.800533 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5224c7bf-aace-4fba-a17f-ecf749f66657 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:35:19.803141 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-5224c7bf-aace-4fba-a17f-ecf749f66657 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:35:19.863734 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5224c7bf-aace-4fba-a17f-ecf749f66657 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:35:19.867650 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e775e5-4789-4dce-a289-ee7636518d51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:35:19.868254 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e775e5-4789-4dce-a289-ee7636518d51 service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:35:19.868538 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e775e5-4789-4dce-a289-ee7636518d51 service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:35:19.868814 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e775e5-4789-4dce-a289-ee7636518d51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:35:19.878142 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e775e5-4789-4dce-a289-ee7636518d51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:35:19.879562 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e775e5-4789-4dce-a289-ee7636518d51 service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:35:19.889787 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1202/2429] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:35:19 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:35:20.149727 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b19839-492c-45e9-9587-94498c57d685 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:35:20.151223 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-b0b19839-492c-45e9-9587-94498c57d685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:35:20.164374 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b19839-492c-45e9-9587-94498c57d685 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:35:20.171215 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7932ba-e8c8-4687-916a-f6fe38b0913b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:35:20.171999 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7932ba-e8c8-4687-916a-f6fe38b0913b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:35:20.172180 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7932ba-e8c8-4687-916a-f6fe38b0913b admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:35:20.172458 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7932ba-e8c8-4687-916a-f6fe38b0913b admin admin] PATH_INFO: `/v3/projects/8c65801234b84dcc93d2e78dbc03dbda` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:35:20.174739 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7932ba-e8c8-4687-916a-f6fe38b0913b admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c65801234b84dcc93d2e78dbc03dbda)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:35:20.176999 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7932ba-e8c8-4687-916a-f6fe38b0913b admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:35:20.227165 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-1e7932ba-e8c8-4687-916a-f6fe38b0913b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c65801234b84dcc93d2e78dbc03dbda', 'request_id': 'req-1e7932ba-e8c8-4687-916a-f6fe38b0913b'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:35:20.234791 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-1e7932ba-e8c8-4687-916a-f6fe38b0913b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c65801234b84dcc93d2e78dbc03dbda', 'request_id': 'req-1e7932ba-e8c8-4687-916a-f6fe38b0913b'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:35:20.241059 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-1e7932ba-e8c8-4687-916a-f6fe38b0913b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c65801234b84dcc93d2e78dbc03dbda', 'request_id': 'req-1e7932ba-e8c8-4687-916a-f6fe38b0913b'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:35:20.251258 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-1e7932ba-e8c8-4687-916a-f6fe38b0913b admin admin] The token cache is being invalidate because project 8c65801234b84dcc93d2e78dbc03dbda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:35:20.251542 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-1e7932ba-e8c8-4687-916a-f6fe38b0913b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:35:20.252152 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-1e7932ba-e8c8-4687-916a-f6fe38b0913b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:35:20.253653 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1228/2430] 10.222.0.102 () {58 vars in 1238 bytes} [Thu Sep 1 17:35:20 2022] DELETE /identity/v3/projects/8c65801234b84dcc93d2e78dbc03dbda => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:35:20.258558 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa7cbe6-e0de-4161-b602-dfd7417ce03f None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:35:20.258864 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa7cbe6-e0de-4161-b602-dfd7417ce03f None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:35:20.259153 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa7cbe6-e0de-4161-b602-dfd7417ce03f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:35:20.280037 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-4fa7cbe6-e0de-4161-b602-dfd7417ce03f None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:35:20.281243 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-4fa7cbe6-e0de-4161-b602-dfd7417ce03f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:35:20.298941 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:35:20.343011 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1203/2431] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:35:20 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:35:20.350904 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b866f5-58ae-441e-a644-dd03854d1953 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:35:20.351193 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b866f5-58ae-441e-a644-dd03854d1953 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:35:20.351469 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b866f5-58ae-441e-a644-dd03854d1953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:35:20.374501 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-b5b866f5-58ae-441e-a644-dd03854d1953 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:35:20.375695 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-b5b866f5-58ae-441e-a644-dd03854d1953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:35:20.383439 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1229/2432] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:35:20 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:35:20.387659 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa7cbe6-e0de-4161-b602-dfd7417ce03f None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:35:20.397957 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa7cbe6-e0de-4161-b602-dfd7417ce03f None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:35:20.402466 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecec8f1-c7c3-42c3-9dee-f0022a222067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:35:20.403077 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecec8f1-c7c3-42c3-9dee-f0022a222067 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:35:20.403357 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecec8f1-c7c3-42c3-9dee-f0022a222067 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:35:20.403621 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecec8f1-c7c3-42c3-9dee-f0022a222067 admin admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:35:20.404286 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecec8f1-c7c3-42c3-9dee-f0022a222067 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:35:20.404600 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecec8f1-c7c3-42c3-9dee-f0022a222067 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:35:20.406239 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecec8f1-c7c3-42c3-9dee-f0022a222067 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:35:20.417468 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1204/2433] 10.222.0.102 () {58 vars in 1158 bytes} [Thu Sep 1 17:35:20 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:35:20.421599 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b866f5-58ae-441e-a644-dd03854d1953 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:35:20.430728 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b866f5-58ae-441e-a644-dd03854d1953 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:35:20.435722 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2946e6-3dd8-420e-a069-b227c60e116c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:35:20.436825 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2946e6-3dd8-420e-a069-b227c60e116c admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:35:20.437103 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2946e6-3dd8-420e-a069-b227c60e116c admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:35:20.437404 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2946e6-3dd8-420e-a069-b227c60e116c admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:35:20.438042 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2946e6-3dd8-420e-a069-b227c60e116c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:35:20.439926 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2946e6-3dd8-420e-a069-b227c60e116c admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:35:20.467790 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1230/2434] 10.222.0.102 () {60 vars in 1158 bytes} [Thu Sep 1 17:35:20 2022] POST /identity/v3/projects => generated 386 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:35:20.471801 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecec8f1-c7c3-42c3-9dee-f0022a222067 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:35:20.481562 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecec8f1-c7c3-42c3-9dee-f0022a222067 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:35:20.539178 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d7a1e7-7d8c-4c54-bc64-6b1d9e7abafd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:35:20.539786 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d7a1e7-7d8c-4c54-bc64-6b1d9e7abafd admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:35:20.540067 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d7a1e7-7d8c-4c54-bc64-6b1d9e7abafd admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:35:20.540344 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d7a1e7-7d8c-4c54-bc64-6b1d9e7abafd admin admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:35:20.540974 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d7a1e7-7d8c-4c54-bc64-6b1d9e7abafd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:35:20.542928 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d7a1e7-7d8c-4c54-bc64-6b1d9e7abafd admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:35:20.565341 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1205/2435] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:35:20 2022] POST /identity/v3/users => generated 294 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:35:20.568471 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2946e6-3dd8-420e-a069-b227c60e116c admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:35:20.578770 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2946e6-3dd8-420e-a069-b227c60e116c admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:35:20.583363 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06903b49-6f81-49b1-ac9b-5dc70e268005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:35:20.583957 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06903b49-6f81-49b1-ac9b-5dc70e268005 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:35:20.584243 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06903b49-6f81-49b1-ac9b-5dc70e268005 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:35:20.584547 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06903b49-6f81-49b1-ac9b-5dc70e268005 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:35:20.585185 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06903b49-6f81-49b1-ac9b-5dc70e268005 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:35:20.586819 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06903b49-6f81-49b1-ac9b-5dc70e268005 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:35:20.604064 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1231/2436] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:35:20 2022] GET /identity/v3/roles => generated 1427 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:35:20.608587 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d7a1e7-7d8c-4c54-bc64-6b1d9e7abafd admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:35:20.618240 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d7a1e7-7d8c-4c54-bc64-6b1d9e7abafd admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:35:20.622790 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd41c5e-e188-45c8-92cf-325bef5c34ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:35:20.623962 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd41c5e-e188-45c8-92cf-325bef5c34ec admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:35:20.624239 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd41c5e-e188-45c8-92cf-325bef5c34ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:35:20.624502 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd41c5e-e188-45c8-92cf-325bef5c34ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:35:20.625126 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd41c5e-e188-45c8-92cf-325bef5c34ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:35:20.626732 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd41c5e-e188-45c8-92cf-325bef5c34ec admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:35:20.636958 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1206/2437] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:35:20 2022] GET /identity/v3/roles => generated 1427 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:35:20.640861 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06903b49-6f81-49b1-ac9b-5dc70e268005 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:35:20.650620 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06903b49-6f81-49b1-ac9b-5dc70e268005 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:35:20.655125 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4add9ec0-8003-4ead-b49c-dbbda5ed8dc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:35:20.655761 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4add9ec0-8003-4ead-b49c-dbbda5ed8dc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:35:20.656036 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4add9ec0-8003-4ead-b49c-dbbda5ed8dc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:35:20.656302 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4add9ec0-8003-4ead-b49c-dbbda5ed8dc0 admin admin] PATH_INFO: `/v3/projects/9644529eb8a94aa39e326871bc7b73e1/users/333dc785d4a84d56bd33a6b305dec2f7/roles/b75cb360925e4ba5a65f14965c103ffd` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:35:20.670900 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4add9ec0-8003-4ead-b49c-dbbda5ed8dc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9644529eb8a94aa39e326871bc7b73e1, user_id=333dc785d4a84d56bd33a6b305dec2f7, role_id=b75cb360925e4ba5a65f14965c103ffd)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:35:20.673554 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4add9ec0-8003-4ead-b49c-dbbda5ed8dc0 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:35:20.686874 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1232/2438] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:35:20 2022] PUT /identity/v3/projects/9644529eb8a94aa39e326871bc7b73e1/users/333dc785d4a84d56bd33a6b305dec2f7/roles/b75cb360925e4ba5a65f14965c103ffd => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:35:20.691745 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48ff917-b075-4f67-8ba4-ee41dc342d2e None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:35:20.692053 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48ff917-b075-4f67-8ba4-ee41dc342d2e None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:35:20.692323 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48ff917-b075-4f67-8ba4-ee41dc342d2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:35:20.711289 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-e48ff917-b075-4f67-8ba4-ee41dc342d2e None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:35:20.712496 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-e48ff917-b075-4f67-8ba4-ee41dc342d2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:35:20.770130 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1207/2439] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:35:20 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:35:20.779714 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4add9ec0-8003-4ead-b49c-dbbda5ed8dc0 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:35:20.781464 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-4add9ec0-8003-4ead-b49c-dbbda5ed8dc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:35:20.804482 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:35:20.843318 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4add9ec0-8003-4ead-b49c-dbbda5ed8dc0 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:35:20.846887 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045d74df-2ee3-483a-808f-088e222b6cee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:35:20.847485 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045d74df-2ee3-483a-808f-088e222b6cee service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:35:20.847765 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045d74df-2ee3-483a-808f-088e222b6cee service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:35:20.848046 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045d74df-2ee3-483a-808f-088e222b6cee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:35:20.857032 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045d74df-2ee3-483a-808f-088e222b6cee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:35:20.858954 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045d74df-2ee3-483a-808f-088e222b6cee service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:35:20.868492 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1233/2440] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:35:20 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:35:24.815114 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd6f409-9451-49f3-85a4-7f6085be8869 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:35:24.815577 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd6f409-9451-49f3-85a4-7f6085be8869 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:35:24.816206 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd6f409-9451-49f3-85a4-7f6085be8869 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:35:24.849132 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-0bd6f409-9451-49f3-85a4-7f6085be8869 None None] MFA Rules not processed for user `333dc785d4a84d56bd33a6b305dec2f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:35:24.851524 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-0bd6f409-9451-49f3-85a4-7f6085be8869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:35:24.870101 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:35:24.919441 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1208/2441] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:35:24 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:35:24.924010 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045d74df-2ee3-483a-808f-088e222b6cee service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:35:24.933989 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045d74df-2ee3-483a-808f-088e222b6cee service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:35:24.944031 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86251484-cdc0-4fe0-883d-1360ea8232cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:35:24.944993 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86251484-cdc0-4fe0-883d-1360ea8232cb admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:35:24.945759 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86251484-cdc0-4fe0-883d-1360ea8232cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:35:24.946222 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86251484-cdc0-4fe0-883d-1360ea8232cb admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:35:24.946882 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86251484-cdc0-4fe0-883d-1360ea8232cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:35:24.948846 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86251484-cdc0-4fe0-883d-1360ea8232cb admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:35:25.007145 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1234/2442] 10.222.0.102 () {60 vars in 1158 bytes} [Thu Sep 1 17:35:24 2022] POST /identity/v3/projects => generated 386 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:35:25.011507 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd6f409-9451-49f3-85a4-7f6085be8869 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:35:25.022902 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd6f409-9451-49f3-85a4-7f6085be8869 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:35:25.088324 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f888867-2182-43bc-b48b-a7f8811400ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:35:25.089193 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f888867-2182-43bc-b48b-a7f8811400ea admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:35:25.089512 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f888867-2182-43bc-b48b-a7f8811400ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:35:25.089789 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f888867-2182-43bc-b48b-a7f8811400ea admin admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:35:25.090439 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f888867-2182-43bc-b48b-a7f8811400ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:35:25.092426 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f888867-2182-43bc-b48b-a7f8811400ea admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:35:25.119820 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1209/2443] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:35:25 2022] POST /identity/v3/users => generated 300 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:35:25.121049 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86251484-cdc0-4fe0-883d-1360ea8232cb admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:35:25.132163 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86251484-cdc0-4fe0-883d-1360ea8232cb admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:35:25.136899 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7789e7ee-2226-4478-90db-6caf73251ea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:35:25.137549 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7789e7ee-2226-4478-90db-6caf73251ea4 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:35:25.137835 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7789e7ee-2226-4478-90db-6caf73251ea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:35:25.138106 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7789e7ee-2226-4478-90db-6caf73251ea4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:35:25.138794 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7789e7ee-2226-4478-90db-6caf73251ea4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:35:25.140392 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7789e7ee-2226-4478-90db-6caf73251ea4 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:35:25.153588 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1235/2444] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:35:25 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:35:25.158014 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f888867-2182-43bc-b48b-a7f8811400ea admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:35:25.168428 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f888867-2182-43bc-b48b-a7f8811400ea admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:35:25.173692 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984fc649-7882-42f6-af63-515fed11cd03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:35:25.174348 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984fc649-7882-42f6-af63-515fed11cd03 admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:35:25.174626 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984fc649-7882-42f6-af63-515fed11cd03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:35:25.175038 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984fc649-7882-42f6-af63-515fed11cd03 admin admin] PATH_INFO: `/v3/projects/230a2d829b2e4013a0bb63abd797c373/users/b17a4e748a5f4d3789c3d8617930632a/roles/36577a6f46d642d4b81805ede9872490` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:35:25.191963 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984fc649-7882-42f6-af63-515fed11cd03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=230a2d829b2e4013a0bb63abd797c373, user_id=b17a4e748a5f4d3789c3d8617930632a, role_id=36577a6f46d642d4b81805ede9872490)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:35:25.194667 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984fc649-7882-42f6-af63-515fed11cd03 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:35:25.213077 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1210/2445] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:35:25 2022] PUT /identity/v3/projects/230a2d829b2e4013a0bb63abd797c373/users/b17a4e748a5f4d3789c3d8617930632a/roles/36577a6f46d642d4b81805ede9872490 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:35:25.217671 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7789e7ee-2226-4478-90db-6caf73251ea4 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:35:25.228277 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7789e7ee-2226-4478-90db-6caf73251ea4 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:35:25.295863 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5e29b9-e5b0-4770-9c15-fc312854e592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:35:25.296797 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5e29b9-e5b0-4770-9c15-fc312854e592 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:35:25.297257 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5e29b9-e5b0-4770-9c15-fc312854e592 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:35:25.297568 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5e29b9-e5b0-4770-9c15-fc312854e592 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:35:25.298226 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5e29b9-e5b0-4770-9c15-fc312854e592 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:35:25.299855 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5e29b9-e5b0-4770-9c15-fc312854e592 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:35:25.311362 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1236/2446] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:35:25 2022] GET /identity/v3/roles => generated 1427 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:35:25.315603 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984fc649-7882-42f6-af63-515fed11cd03 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:35:25.327067 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984fc649-7882-42f6-af63-515fed11cd03 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:35:25.332012 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9b3849-98bf-4dbb-b22c-9b7da36e6f09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:35:25.332684 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9b3849-98bf-4dbb-b22c-9b7da36e6f09 admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:35:25.332969 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9b3849-98bf-4dbb-b22c-9b7da36e6f09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:35:25.333243 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9b3849-98bf-4dbb-b22c-9b7da36e6f09 admin admin] PATH_INFO: `/v3/projects/230a2d829b2e4013a0bb63abd797c373/users/b17a4e748a5f4d3789c3d8617930632a/roles/36577a6f46d642d4b81805ede9872490` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:35:25.335307 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9b3849-98bf-4dbb-b22c-9b7da36e6f09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=230a2d829b2e4013a0bb63abd797c373, user_id=b17a4e748a5f4d3789c3d8617930632a, role_id=36577a6f46d642d4b81805ede9872490)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:35:25.338055 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9b3849-98bf-4dbb-b22c-9b7da36e6f09 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:35:25.350037 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1211/2447] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:35:25 2022] PUT /identity/v3/projects/230a2d829b2e4013a0bb63abd797c373/users/b17a4e748a5f4d3789c3d8617930632a/roles/36577a6f46d642d4b81805ede9872490 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:35:30.025762 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f701a73-99e1-47c7-bddd-e63cbc41afe0 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:35:30.026265 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f701a73-99e1-47c7-bddd-e63cbc41afe0 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:35:30.026480 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f701a73-99e1-47c7-bddd-e63cbc41afe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:35:30.065506 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-2f701a73-99e1-47c7-bddd-e63cbc41afe0 None None] MFA Rules not processed for user `b17a4e748a5f4d3789c3d8617930632a`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:35:30.066745 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-2f701a73-99e1-47c7-bddd-e63cbc41afe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:35:30.158791 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1237/2448] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:35:30 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:35:30.171590 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9b3849-98bf-4dbb-b22c-9b7da36e6f09 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:35:30.173412 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-ae9b3849-98bf-4dbb-b22c-9b7da36e6f09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:35:30.231305 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9b3849-98bf-4dbb-b22c-9b7da36e6f09 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:35:30.235579 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b487e87-9b68-4426-9d2d-0e4d48687238 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '039683b8d1d146c6add5ffa235d06f91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:35:30.236188 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b487e87-9b68-4426-9d2d-0e4d48687238 service glance] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:35:30.236497 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b487e87-9b68-4426-9d2d-0e4d48687238 service glance] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:35:30.236781 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b487e87-9b68-4426-9d2d-0e4d48687238 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:35:30.262594 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:35:30.312727 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b487e87-9b68-4426-9d2d-0e4d48687238 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:35:30.314428 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b487e87-9b68-4426-9d2d-0e4d48687238 service glance] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:35:30.323941 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1212/2449] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:35:30 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:35:30.501822 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03d4c39-0038-425c-9860-b90a67b80772 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:35:30.505235 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03d4c39-0038-425c-9860-b90a67b80772 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:35:30.507413 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03d4c39-0038-425c-9860-b90a67b80772 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:35:30.539254 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-d03d4c39-0038-425c-9860-b90a67b80772 None None] MFA Rules not processed for user `575a76b99ea945efb1024fbb3dc81aaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:35:30.540680 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-d03d4c39-0038-425c-9860-b90a67b80772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:35:30.547316 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:35:30.607825 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1238/2450] 10.222.0.102 () {58 vars in 1014 bytes} [Thu Sep 1 17:35:30 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:35:30.678002 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b487e87-9b68-4426-9d2d-0e4d48687238 service glance] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:35:30.679714 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-5b487e87-9b68-4426-9d2d-0e4d48687238 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:35:30.755779 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b487e87-9b68-4426-9d2d-0e4d48687238 service glance] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:35:30.763570 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebbf9ac-ca36-4059-9b06-877d5f7ae3b1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd55edf6cd1804006aa8269b4c6f80aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:35:30.764213 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebbf9ac-ca36-4059-9b06-877d5f7ae3b1 service swift] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:35:30.764491 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebbf9ac-ca36-4059-9b06-877d5f7ae3b1 service swift] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:35:30.764759 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebbf9ac-ca36-4059-9b06-877d5f7ae3b1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:35:30.772903 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebbf9ac-ca36-4059-9b06-877d5f7ae3b1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:35:30.774522 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebbf9ac-ca36-4059-9b06-877d5f7ae3b1 service swift] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:35:30.783001 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1213/2451] 10.222.0.102 () {60 vars in 1389 bytes} [Thu Sep 1 17:35:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 17:35:39.457387 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f2128b-2c55-49b9-bdfa-5b734f25d123 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:35:39.457966 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f2128b-2c55-49b9-bdfa-5b734f25d123 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:35:39.457966 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f2128b-2c55-49b9-bdfa-5b734f25d123 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:35:39.476812 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-86f2128b-2c55-49b9-bdfa-5b734f25d123 None None] MFA Rules not processed for user `333dc785d4a84d56bd33a6b305dec2f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:35:39.478043 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-86f2128b-2c55-49b9-bdfa-5b734f25d123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:35:39.485519 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1239/2452] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:35:39 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:35:40.779124 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=48 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:35:41.984500 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8a7ef6-0d18-46fe-a011-475e243edbbf None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:35:41.985092 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8a7ef6-0d18-46fe-a011-475e243edbbf None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:35:41.985092 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8a7ef6-0d18-46fe-a011-475e243edbbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:35:42.003675 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-cb8a7ef6-0d18-46fe-a011-475e243edbbf None None] MFA Rules not processed for user `575a76b99ea945efb1024fbb3dc81aaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:35:42.004855 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-cb8a7ef6-0d18-46fe-a011-475e243edbbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:35:42.012388 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1214/2453] 10.222.0.102 () {58 vars in 1014 bytes} [Thu Sep 1 17:35:41 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:35:42.024358 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f2128b-2c55-49b9-bdfa-5b734f25d123 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:35:42.037460 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f2128b-2c55-49b9-bdfa-5b734f25d123 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:35:42.056308 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f3279f-1d59-48b8-9900-f4206e76e698 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd55edf6cd1804006aa8269b4c6f80aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:35:42.056308 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f3279f-1d59-48b8-9900-f4206e76e698 service swift] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:35:42.056308 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f3279f-1d59-48b8-9900-f4206e76e698 service swift] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:35:42.056308 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f3279f-1d59-48b8-9900-f4206e76e698 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:35:42.066804 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f3279f-1d59-48b8-9900-f4206e76e698 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:35:42.068619 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f3279f-1d59-48b8-9900-f4206e76e698 service swift] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:35:42.078284 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1240/2454] 10.222.0.102 () {60 vars in 1389 bytes} [Thu Sep 1 17:35:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 17:35:51.998428 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=3 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:35:52.074264 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=29 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:35:59.106594 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f3f2f2-c88e-48a2-86dd-f727ee04889a None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:35:59.107296 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f3f2f2-c88e-48a2-86dd-f727ee04889a None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:35:59.107519 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f3f2f2-c88e-48a2-86dd-f727ee04889a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:35:59.126877 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-27f3f2f2-c88e-48a2-86dd-f727ee04889a None None] MFA Rules not processed for user `575a76b99ea945efb1024fbb3dc81aaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:35:59.128084 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-27f3f2f2-c88e-48a2-86dd-f727ee04889a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:35:59.135352 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1215/2455] 10.222.0.102 () {58 vars in 1014 bytes} [Thu Sep 1 17:35:59 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:35:59.147905 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f3279f-1d59-48b8-9900-f4206e76e698 service swift] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:35:59.159001 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f3279f-1d59-48b8-9900-f4206e76e698 service swift] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:35:59.166282 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6290ae48-06af-45f9-a63c-4af3456c4105 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd55edf6cd1804006aa8269b4c6f80aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:35:59.166954 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6290ae48-06af-45f9-a63c-4af3456c4105 service swift] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:35:59.167239 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6290ae48-06af-45f9-a63c-4af3456c4105 service swift] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:35:59.167519 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6290ae48-06af-45f9-a63c-4af3456c4105 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:35:59.176883 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6290ae48-06af-45f9-a63c-4af3456c4105 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:35:59.178506 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6290ae48-06af-45f9-a63c-4af3456c4105 service swift] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:35:59.186659 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1241/2456] 10.222.0.102 () {60 vars in 1389 bytes} [Thu Sep 1 17:35:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 17:36:09.121381 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=3 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:36:09.183358 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=6 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:36:10.557661 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f3f2f2-c88e-48a2-86dd-f727ee04889a None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:10.623401 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f3f2f2-c88e-48a2-86dd-f727ee04889a None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:10.628302 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34e74a0-0ae9-47bc-9269-66d0ccab9b51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:10.629100 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34e74a0-0ae9-47bc-9269-66d0ccab9b51 service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:10.629409 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34e74a0-0ae9-47bc-9269-66d0ccab9b51 service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:10.629680 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34e74a0-0ae9-47bc-9269-66d0ccab9b51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:10.631585 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-e34e74a0-0ae9-47bc-9269-66d0ccab9b51 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:10.718022 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34e74a0-0ae9-47bc-9269-66d0ccab9b51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:10.719476 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34e74a0-0ae9-47bc-9269-66d0ccab9b51 service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:10.733176 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1216/2457] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:36:10 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:36:10.980451 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6290ae48-06af-45f9-a63c-4af3456c4105 service swift] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:10.982245 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-6290ae48-06af-45f9-a63c-4af3456c4105 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:10.996218 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6290ae48-06af-45f9-a63c-4af3456c4105 service swift] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:11.001586 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87c9998-3558-4971-97ad-31733cbd003d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:11.002285 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87c9998-3558-4971-97ad-31733cbd003d service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:11.002552 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87c9998-3558-4971-97ad-31733cbd003d service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:11.005822 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87c9998-3558-4971-97ad-31733cbd003d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:11.008173 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-f87c9998-3558-4971-97ad-31733cbd003d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:11.024951 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87c9998-3558-4971-97ad-31733cbd003d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:11.027801 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87c9998-3558-4971-97ad-31733cbd003d service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:11.049431 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1242/2458] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:36:10 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:36:16.543518 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c353f522-da30-4329-b4f7-b8f6d2815c4e None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:16.544086 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c353f522-da30-4329-b4f7-b8f6d2815c4e None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:16.544207 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c353f522-da30-4329-b4f7-b8f6d2815c4e None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:16.545525 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1217/2459] 10.222.0.102 () {54 vars in 926 bytes} [Thu Sep 1 17:36:16 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:36:16.552349 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6141a189-ca65-48fe-86e7-030de5d8241b None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:16.552803 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6141a189-ca65-48fe-86e7-030de5d8241b None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:16.552898 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6141a189-ca65-48fe-86e7-030de5d8241b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:16.572696 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-6141a189-ca65-48fe-86e7-030de5d8241b None None] MFA Rules not processed for user `5f8293316baf4774b62fc11d55d50865`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:36:16.573918 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-6141a189-ca65-48fe-86e7-030de5d8241b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:16.583680 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1243/2460] 10.222.0.102 () {58 vars in 1023 bytes} [Thu Sep 1 17:36:16 2022] POST /identity/v3/auth/tokens => generated 3382 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:36:16.596355 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c353f522-da30-4329-b4f7-b8f6d2815c4e None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:16.607126 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c353f522-da30-4329-b4f7-b8f6d2815c4e None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:16.610582 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d7a9fa-f8e7-4d0f-94e1-c3ad16af8078 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '039683b8d1d146c6add5ffa235d06f91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:16.611166 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d7a9fa-f8e7-4d0f-94e1-c3ad16af8078 service glance] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:16.611441 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d7a9fa-f8e7-4d0f-94e1-c3ad16af8078 service glance] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:16.611708 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d7a9fa-f8e7-4d0f-94e1-c3ad16af8078 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:16.622902 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d7a9fa-f8e7-4d0f-94e1-c3ad16af8078 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:16.624316 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d7a9fa-f8e7-4d0f-94e1-c3ad16af8078 service glance] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:16.635234 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1218/2461] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:36:16 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:36:16.705168 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48387fb-6b73-4bbf-a10d-c2053ca6b121 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:16.705486 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48387fb-6b73-4bbf-a10d-c2053ca6b121 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:16.705777 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48387fb-6b73-4bbf-a10d-c2053ca6b121 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:16.707171 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1244/2462] 10.222.0.102 () {54 vars in 926 bytes} [Thu Sep 1 17:36:16 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:36:16.711982 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c40f449-863c-4edc-9a71-65ca9998b0a0 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:16.712265 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c40f449-863c-4edc-9a71-65ca9998b0a0 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:16.712536 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c40f449-863c-4edc-9a71-65ca9998b0a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:16.732476 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-4c40f449-863c-4edc-9a71-65ca9998b0a0 None None] MFA Rules not processed for user `7c7d3594fdab4191be1d841af43a5c79`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:36:16.733695 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-4c40f449-863c-4edc-9a71-65ca9998b0a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:16.740137 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1219/2463] 10.222.0.102 () {58 vars in 1023 bytes} [Thu Sep 1 17:36:16 2022] POST /identity/v3/auth/tokens => generated 3131 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:36:16.751420 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48387fb-6b73-4bbf-a10d-c2053ca6b121 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:16.762925 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48387fb-6b73-4bbf-a10d-c2053ca6b121 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:16.766558 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2ed7d4-8963-43e9-86ab-5562149af140 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:16.767141 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2ed7d4-8963-43e9-86ab-5562149af140 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:16.767419 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2ed7d4-8963-43e9-86ab-5562149af140 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:16.767688 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2ed7d4-8963-43e9-86ab-5562149af140 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:16.774556 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2ed7d4-8963-43e9-86ab-5562149af140 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:16.775973 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2ed7d4-8963-43e9-86ab-5562149af140 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:16.784582 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1245/2464] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:36:16 2022] GET /identity/v3/auth/tokens => generated 3131 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:36:16.868683 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d38725-229e-416e-ab47-d5482640e331 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:16.869001 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d38725-229e-416e-ab47-d5482640e331 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:16.869257 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d38725-229e-416e-ab47-d5482640e331 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:16.870725 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1220/2465] 10.222.0.102 () {54 vars in 926 bytes} [Thu Sep 1 17:36:16 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:36:16.875602 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b33e9e3-1599-400a-92c7-3c3af3371f19 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:16.875895 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b33e9e3-1599-400a-92c7-3c3af3371f19 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:16.876173 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b33e9e3-1599-400a-92c7-3c3af3371f19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:16.896272 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-6b33e9e3-1599-400a-92c7-3c3af3371f19 None None] MFA Rules not processed for user `5f8293316baf4774b62fc11d55d50865`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:36:16.897614 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-6b33e9e3-1599-400a-92c7-3c3af3371f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:16.910702 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1246/2466] 10.222.0.102 () {58 vars in 1023 bytes} [Thu Sep 1 17:36:16 2022] POST /identity/v3/auth/tokens => generated 3382 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:36:16.921843 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d38725-229e-416e-ab47-d5482640e331 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:16.937089 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d38725-229e-416e-ab47-d5482640e331 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:16.943639 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b750f6ec-8da2-4647-9f50-ba25dbcacbff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:16.944223 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b750f6ec-8da2-4647-9f50-ba25dbcacbff service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:16.944500 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b750f6ec-8da2-4647-9f50-ba25dbcacbff service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:16.944774 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b750f6ec-8da2-4647-9f50-ba25dbcacbff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:16.956042 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b750f6ec-8da2-4647-9f50-ba25dbcacbff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:16.957495 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b750f6ec-8da2-4647-9f50-ba25dbcacbff service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:16.968353 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1221/2467] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:36:16 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 17:36:18.874870 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b33e9e3-1599-400a-92c7-3c3af3371f19 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:18.876264 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-6b33e9e3-1599-400a-92c7-3c3af3371f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:18.890427 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b33e9e3-1599-400a-92c7-3c3af3371f19 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:18.896044 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61219fd4-c488-4165-a730-b85e12bec872 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f8293316baf4774b62fc11d55d50865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:18.896628 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61219fd4-c488-4165-a730-b85e12bec872 service nova] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:18.896912 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61219fd4-c488-4165-a730-b85e12bec872 service nova] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:18.897184 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61219fd4-c488-4165-a730-b85e12bec872 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:18.898845 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-61219fd4-c488-4165-a730-b85e12bec872 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:18.960526 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61219fd4-c488-4165-a730-b85e12bec872 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:18.962423 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61219fd4-c488-4165-a730-b85e12bec872 service nova] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:18.971954 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1247/2468] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:36:18 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:36:23.936903 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b750f6ec-8da2-4647-9f50-ba25dbcacbff service neutron] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:23.945357 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b750f6ec-8da2-4647-9f50-ba25dbcacbff service neutron] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:23.948934 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b2a945-4ecc-47de-bb0c-d0a9b585e8f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:23.949576 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b2a945-4ecc-47de-bb0c-d0a9b585e8f4 service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:23.949854 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b2a945-4ecc-47de-bb0c-d0a9b585e8f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:23.950125 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b2a945-4ecc-47de-bb0c-d0a9b585e8f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:23.958890 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b2a945-4ecc-47de-bb0c-d0a9b585e8f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:23.960302 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b2a945-4ecc-47de-bb0c-d0a9b585e8f4 service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:23.970285 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1222/2469] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:36:23 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:36:28.967531 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=46 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:36:31.709546 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61219fd4-c488-4165-a730-b85e12bec872 service nova] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:31.719247 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61219fd4-c488-4165-a730-b85e12bec872 service nova] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:31.722807 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4986b3-edec-426b-b192-95b0e2044141 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:31.723397 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4986b3-edec-426b-b192-95b0e2044141 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:31.723675 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4986b3-edec-426b-b192-95b0e2044141 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:31.723958 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4986b3-edec-426b-b192-95b0e2044141 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:31.725662 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-0c4986b3-edec-426b-b192-95b0e2044141 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:31.735574 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4986b3-edec-426b-b192-95b0e2044141 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:31.736975 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4986b3-edec-426b-b192-95b0e2044141 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:31.744728 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1248/2470] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:36:31 2022] GET /identity/v3/auth/tokens => generated 3131 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:36:32.739298 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67238e15-891f-40f0-b82f-a230ca41df89 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:32.739766 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67238e15-891f-40f0-b82f-a230ca41df89 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:32.739880 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67238e15-891f-40f0-b82f-a230ca41df89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:32.758919 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-67238e15-891f-40f0-b82f-a230ca41df89 None None] MFA Rules not processed for user `575a76b99ea945efb1024fbb3dc81aaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:36:32.760445 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-67238e15-891f-40f0-b82f-a230ca41df89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:32.768883 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1223/2471] 10.222.0.102 () {58 vars in 1014 bytes} [Thu Sep 1 17:36:32 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:36:32.791992 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4986b3-edec-426b-b192-95b0e2044141 service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:32.804049 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4986b3-edec-426b-b192-95b0e2044141 service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:32.809227 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdccac7-7b19-444d-823d-af533b983b71 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd55edf6cd1804006aa8269b4c6f80aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:32.810450 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdccac7-7b19-444d-823d-af533b983b71 service swift] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:32.810450 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdccac7-7b19-444d-823d-af533b983b71 service swift] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:32.810450 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdccac7-7b19-444d-823d-af533b983b71 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:32.818987 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdccac7-7b19-444d-823d-af533b983b71 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:32.820589 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdccac7-7b19-444d-823d-af533b983b71 service swift] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:32.829433 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1249/2472] 10.222.0.102 () {60 vars in 1389 bytes} [Thu Sep 1 17:36:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 17:36:42.753949 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=78 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:36:42.826058 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=12 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:36:48.826205 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67238e15-891f-40f0-b82f-a230ca41df89 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:48.839303 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67238e15-891f-40f0-b82f-a230ca41df89 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:48.844795 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409b976d-7b3c-47fa-bb8d-1afef65b19ac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f8293316baf4774b62fc11d55d50865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:48.845428 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409b976d-7b3c-47fa-bb8d-1afef65b19ac service nova] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:48.845709 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409b976d-7b3c-47fa-bb8d-1afef65b19ac service nova] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:48.845982 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409b976d-7b3c-47fa-bb8d-1afef65b19ac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:48.847731 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-409b976d-7b3c-47fa-bb8d-1afef65b19ac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:48.905062 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409b976d-7b3c-47fa-bb8d-1afef65b19ac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:48.906678 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409b976d-7b3c-47fa-bb8d-1afef65b19ac service nova] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:48.915735 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1224/2473] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:36:48 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:36:58.911497 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=28 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:01.426137 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdccac7-7b19-444d-823d-af533b983b71 service swift] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:01.427656 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-cfdccac7-7b19-444d-823d-af533b983b71 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:01.444987 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdccac7-7b19-444d-823d-af533b983b71 service swift] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:01.451111 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9521314d-43c2-44e8-973e-3288ea23648d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f8293316baf4774b62fc11d55d50865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:01.451718 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9521314d-43c2-44e8-973e-3288ea23648d service nova] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:01.452001 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9521314d-43c2-44e8-973e-3288ea23648d service nova] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:01.452272 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9521314d-43c2-44e8-973e-3288ea23648d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:01.454016 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-9521314d-43c2-44e8-973e-3288ea23648d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:01.466718 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9521314d-43c2-44e8-973e-3288ea23648d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:01.468310 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9521314d-43c2-44e8-973e-3288ea23648d service nova] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:01.479646 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1250/2474] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:37:01 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:37:04.232748 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e19df2-90f5-46cf-8495-3286dd50d579 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:04.233245 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e19df2-90f5-46cf-8495-3286dd50d579 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:04.233391 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e19df2-90f5-46cf-8495-3286dd50d579 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:04.254268 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-a7e19df2-90f5-46cf-8495-3286dd50d579 None None] MFA Rules not processed for user `575a76b99ea945efb1024fbb3dc81aaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:37:04.255497 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-a7e19df2-90f5-46cf-8495-3286dd50d579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:04.270289 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1225/2475] 10.222.0.102 () {58 vars in 972 bytes} [Thu Sep 1 17:37:04 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:37:04.285197 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9521314d-43c2-44e8-973e-3288ea23648d service nova] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:04.297053 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9521314d-43c2-44e8-973e-3288ea23648d service nova] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:04.303961 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1199c8f8-cb0a-453d-b813-42cb02e49c76 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd55edf6cd1804006aa8269b4c6f80aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:04.305009 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1199c8f8-cb0a-453d-b813-42cb02e49c76 service swift] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:04.305577 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1199c8f8-cb0a-453d-b813-42cb02e49c76 service swift] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:04.306074 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1199c8f8-cb0a-453d-b813-42cb02e49c76 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:04.317279 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1199c8f8-cb0a-453d-b813-42cb02e49c76 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:04.319094 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1199c8f8-cb0a-453d-b813-42cb02e49c76 service swift] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:04.328170 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1251/2476] 10.222.0.102 () {60 vars in 1389 bytes} [Thu Sep 1 17:37:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 17:37:14.247216 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=3 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:14.324721 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=12 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:26.809981 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d5c8bf-154a-427c-859c-bc44f4161a2f None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:26.810533 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d5c8bf-154a-427c-859c-bc44f4161a2f None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:26.810533 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d5c8bf-154a-427c-859c-bc44f4161a2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:26.829051 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-a4d5c8bf-154a-427c-859c-bc44f4161a2f None None] MFA Rules not processed for user `d364c74791fc4cd4af6773433a7a13cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:37:26.830276 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-a4d5c8bf-154a-427c-859c-bc44f4161a2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:26.837287 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1226/2477] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:37:26 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:37:34.269231 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1199c8f8-cb0a-453d-b813-42cb02e49c76 service swift] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:34.283327 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1199c8f8-cb0a-453d-b813-42cb02e49c76 service swift] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:34.303930 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2563116b-261f-4c89-9ed9-71be8558b499 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f8293316baf4774b62fc11d55d50865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:34.304548 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2563116b-261f-4c89-9ed9-71be8558b499 service nova] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:34.304830 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2563116b-261f-4c89-9ed9-71be8558b499 service nova] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:34.305105 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2563116b-261f-4c89-9ed9-71be8558b499 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:34.307540 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-2563116b-261f-4c89-9ed9-71be8558b499 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:34.329012 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2563116b-261f-4c89-9ed9-71be8558b499 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:34.330653 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2563116b-261f-4c89-9ed9-71be8558b499 service nova] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:34.342546 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1252/2478] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:37:34 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:37:36.824309 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=3 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:44.338570 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=6 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:58.872420 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f6b414-d805-4566-a85e-ada8a607139b None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:58.872932 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f6b414-d805-4566-a85e-ada8a607139b None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:58.873016 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f6b414-d805-4566-a85e-ada8a607139b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:58.893942 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-42f6b414-d805-4566-a85e-ada8a607139b None None] MFA Rules not processed for user `8f59383e2f6c4da8b683d718f62d25fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:37:58.895192 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-42f6b414-d805-4566-a85e-ada8a607139b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:58.955377 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1227/2479] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:37:58 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:37:59.503232 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2563116b-261f-4c89-9ed9-71be8558b499 service nova] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:59.512696 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2563116b-261f-4c89-9ed9-71be8558b499 service nova] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:59.516423 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a091cc9d-5082-48bb-a718-13fbde7093b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:59.517032 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a091cc9d-5082-48bb-a718-13fbde7093b3 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:59.517331 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a091cc9d-5082-48bb-a718-13fbde7093b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:59.517681 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a091cc9d-5082-48bb-a718-13fbde7093b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:59.519400 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-a091cc9d-5082-48bb-a718-13fbde7093b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:59.529602 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a091cc9d-5082-48bb-a718-13fbde7093b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:59.531053 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a091cc9d-5082-48bb-a718-13fbde7093b3 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:59.541088 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1253/2480] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:37:59 2022] GET /identity/v3/auth/tokens => generated 3131 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:37:59.977901 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f6b414-d805-4566-a85e-ada8a607139b None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:59.979529 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-42f6b414-d805-4566-a85e-ada8a607139b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:00.057718 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f6b414-d805-4566-a85e-ada8a607139b None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:00.063168 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd95c1c-2780-4ae8-a4dd-17dd10948b16 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '134ef6039cef468bae0e3f3be79b8073', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:00.063805 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd95c1c-2780-4ae8-a4dd-17dd10948b16 service placement] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:00.064080 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd95c1c-2780-4ae8-a4dd-17dd10948b16 service placement] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:00.064349 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd95c1c-2780-4ae8-a4dd-17dd10948b16 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:00.066149 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-5cd95c1c-2780-4ae8-a4dd-17dd10948b16 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:00.078045 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd95c1c-2780-4ae8-a4dd-17dd10948b16 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:00.079606 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd95c1c-2780-4ae8-a4dd-17dd10948b16 service placement] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:00.089543 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1228/2481] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:37:59 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 17:38:09.535912 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=6 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:38:10.086056 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=57 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:38:14.151184 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a091cc9d-5082-48bb-a718-13fbde7093b3 service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:14.164775 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a091cc9d-5082-48bb-a718-13fbde7093b3 service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:14.170316 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670b0c4a-1c36-42a1-bf5a-54ff4057dba6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f8293316baf4774b62fc11d55d50865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:14.170914 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670b0c4a-1c36-42a1-bf5a-54ff4057dba6 service nova] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:14.171198 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670b0c4a-1c36-42a1-bf5a-54ff4057dba6 service nova] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:14.171477 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670b0c4a-1c36-42a1-bf5a-54ff4057dba6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:14.173192 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-670b0c4a-1c36-42a1-bf5a-54ff4057dba6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:14.231032 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670b0c4a-1c36-42a1-bf5a-54ff4057dba6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:14.232638 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670b0c4a-1c36-42a1-bf5a-54ff4057dba6 service nova] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:14.241911 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1254/2482] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:38:14 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:38:24.237644 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=28 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:38:35.028512 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdac77b-3dcd-40fa-8851-9fc0d655d9e8 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:35.029083 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdac77b-3dcd-40fa-8851-9fc0d655d9e8 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:35.029205 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdac77b-3dcd-40fa-8851-9fc0d655d9e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:35.052792 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-3fdac77b-3dcd-40fa-8851-9fc0d655d9e8 None None] MFA Rules not processed for user `26bf17f1fb8948008fed82f4ba33bf66`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:38:35.054554 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-3fdac77b-3dcd-40fa-8851-9fc0d655d9e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:35.061790 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1229/2483] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:38:35 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:38:45.044536 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=3 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:38:50.857380 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670b0c4a-1c36-42a1-bf5a-54ff4057dba6 service nova] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:50.869033 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670b0c4a-1c36-42a1-bf5a-54ff4057dba6 service nova] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:50.874572 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42605c85-adfa-4a11-befb-1cdfcc123420 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f8293316baf4774b62fc11d55d50865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:50.875217 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42605c85-adfa-4a11-befb-1cdfcc123420 service nova] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:50.875498 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42605c85-adfa-4a11-befb-1cdfcc123420 service nova] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:50.875770 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42605c85-adfa-4a11-befb-1cdfcc123420 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:50.877568 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-42605c85-adfa-4a11-befb-1cdfcc123420 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:50.889656 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42605c85-adfa-4a11-befb-1cdfcc123420 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:50.891240 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42605c85-adfa-4a11-befb-1cdfcc123420 service nova] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:50.902535 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1255/2484] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:38:50 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:39:00.897857 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=6 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:39:11.144058 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdac77b-3dcd-40fa-8851-9fc0d655d9e8 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:11.164428 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdac77b-3dcd-40fa-8851-9fc0d655d9e8 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:11.169096 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63531349-d0a7-4f88-958e-3f945ec250e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:11.169710 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63531349-d0a7-4f88-958e-3f945ec250e8 service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:11.169987 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63531349-d0a7-4f88-958e-3f945ec250e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:11.170287 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63531349-d0a7-4f88-958e-3f945ec250e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:11.171964 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-63531349-d0a7-4f88-958e-3f945ec250e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:39:11.282132 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63531349-d0a7-4f88-958e-3f945ec250e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:11.283591 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63531349-d0a7-4f88-958e-3f945ec250e8 service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:11.304719 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1230/2485] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:39:11 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:39:14.535367 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42605c85-adfa-4a11-befb-1cdfcc123420 service nova] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:14.599945 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63531349-d0a7-4f88-958e-3f945ec250e8 service neutron] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:14.601751 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-63531349-d0a7-4f88-958e-3f945ec250e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:39:14.615248 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63531349-d0a7-4f88-958e-3f945ec250e8 service neutron] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:14.621425 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0bf293-bb3f-43c8-b2dc-1c12e27f3045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:14.622019 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0bf293-bb3f-43c8-b2dc-1c12e27f3045 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:14.622302 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0bf293-bb3f-43c8-b2dc-1c12e27f3045 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:14.622580 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0bf293-bb3f-43c8-b2dc-1c12e27f3045 admin admin] PATH_INFO: `/v3/users/d364c74791fc4cd4af6773433a7a13cc` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:14.623842 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0bf293-bb3f-43c8-b2dc-1c12e27f3045 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d364c74791fc4cd4af6773433a7a13cc)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:14.624678 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42605c85-adfa-4a11-befb-1cdfcc123420 service nova] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:14.625870 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0bf293-bb3f-43c8-b2dc-1c12e27f3045 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:14.633113 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f1ce7e-fadc-42df-8e82-bf1db7b7bab9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '134ef6039cef468bae0e3f3be79b8073', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:14.633769 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f1ce7e-fadc-42df-8e82-bf1db7b7bab9 service placement] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:14.634045 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f1ce7e-fadc-42df-8e82-bf1db7b7bab9 service placement] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:14.634497 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f1ce7e-fadc-42df-8e82-bf1db7b7bab9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:14.636250 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-f9f1ce7e-fadc-42df-8e82-bf1db7b7bab9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:39:14.664187 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f1ce7e-fadc-42df-8e82-bf1db7b7bab9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:14.665855 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f1ce7e-fadc-42df-8e82-bf1db7b7bab9 service placement] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:14.675892 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1256/2486] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:39:14 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:39:14.685708 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-ce0bf293-bb3f-43c8-b2dc-1c12e27f3045 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd364c74791fc4cd4af6773433a7a13cc'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:14.686383 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-ce0bf293-bb3f-43c8-b2dc-1c12e27f3045 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd364c74791fc4cd4af6773433a7a13cc'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:14.699606 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-ce0bf293-bb3f-43c8-b2dc-1c12e27f3045 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd364c74791fc4cd4af6773433a7a13cc'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:14.708111 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-ce0bf293-bb3f-43c8-b2dc-1c12e27f3045 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd364c74791fc4cd4af6773433a7a13cc'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:14.708828 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-ce0bf293-bb3f-43c8-b2dc-1c12e27f3045 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd364c74791fc4cd4af6773433a7a13cc'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:14.716603 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:39:14.722863 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1231/2487] 10.222.0.102 () {58 vars in 1229 bytes} [Thu Sep 1 17:39:14 2022] DELETE /identity/v3/users/d364c74791fc4cd4af6773433a7a13cc => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:39:14.897330 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f1ce7e-fadc-42df-8e82-bf1db7b7bab9 service placement] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:14.909955 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f1ce7e-fadc-42df-8e82-bf1db7b7bab9 service placement] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:14.967800 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3af37d-2248-4781-8cc8-9da45d77047f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:14.968358 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3af37d-2248-4781-8cc8-9da45d77047f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:14.968641 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3af37d-2248-4781-8cc8-9da45d77047f admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:14.968919 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3af37d-2248-4781-8cc8-9da45d77047f admin admin] PATH_INFO: `/v3/projects/79c90aaff0a44b278807167fdbd38e48` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:14.970067 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3af37d-2248-4781-8cc8-9da45d77047f admin admin] RBAC: Authorizing `identity:delete_project(project_id=79c90aaff0a44b278807167fdbd38e48)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:14.971982 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3af37d-2248-4781-8cc8-9da45d77047f admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:15.017661 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-ee3af37d-2248-4781-8cc8-9da45d77047f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79c90aaff0a44b278807167fdbd38e48', 'request_id': 'req-ee3af37d-2248-4781-8cc8-9da45d77047f'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:15.024870 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-ee3af37d-2248-4781-8cc8-9da45d77047f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79c90aaff0a44b278807167fdbd38e48', 'request_id': 'req-ee3af37d-2248-4781-8cc8-9da45d77047f'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:15.030932 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-ee3af37d-2248-4781-8cc8-9da45d77047f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79c90aaff0a44b278807167fdbd38e48', 'request_id': 'req-ee3af37d-2248-4781-8cc8-9da45d77047f'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:15.041693 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-ee3af37d-2248-4781-8cc8-9da45d77047f admin admin] The token cache is being invalidate because project 79c90aaff0a44b278807167fdbd38e48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:39:15.041986 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-ee3af37d-2248-4781-8cc8-9da45d77047f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:15.042619 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-ee3af37d-2248-4781-8cc8-9da45d77047f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:15.044073 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1257/2488] 10.222.0.102 () {58 vars in 1238 bytes} [Thu Sep 1 17:39:14 2022] DELETE /identity/v3/projects/79c90aaff0a44b278807167fdbd38e48 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:39:15.048967 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053c63f1-3826-4a0b-a2fe-1a8891b3791a None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:15.049257 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053c63f1-3826-4a0b-a2fe-1a8891b3791a None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:15.049576 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053c63f1-3826-4a0b-a2fe-1a8891b3791a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:15.107632 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-053c63f1-3826-4a0b-a2fe-1a8891b3791a None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:39:15.108845 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-053c63f1-3826-4a0b-a2fe-1a8891b3791a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:39:15.167930 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1232/2489] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:39:15 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:39:15.177514 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d7b34d-00ae-4f74-8bd6-cf046ee9b44b None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:15.177806 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d7b34d-00ae-4f74-8bd6-cf046ee9b44b None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:15.178083 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d7b34d-00ae-4f74-8bd6-cf046ee9b44b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:15.197471 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-41d7b34d-00ae-4f74-8bd6-cf046ee9b44b None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:39:15.199553 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-41d7b34d-00ae-4f74-8bd6-cf046ee9b44b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:39:15.208044 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1258/2490] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:39:15 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:39:15.212139 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053c63f1-3826-4a0b-a2fe-1a8891b3791a None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:15.221855 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053c63f1-3826-4a0b-a2fe-1a8891b3791a None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:15.226829 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9169c6-c037-470e-958a-583da7f74ebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:15.227419 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9169c6-c037-470e-958a-583da7f74ebe admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:15.227703 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9169c6-c037-470e-958a-583da7f74ebe admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:15.227973 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9169c6-c037-470e-958a-583da7f74ebe admin admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:15.228615 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9169c6-c037-470e-958a-583da7f74ebe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:39:15.228921 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9169c6-c037-470e-958a-583da7f74ebe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:15.230612 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9169c6-c037-470e-958a-583da7f74ebe admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:15.242505 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1233/2491] 10.222.0.102 () {58 vars in 1158 bytes} [Thu Sep 1 17:39:15 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:39:15.246511 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d7b34d-00ae-4f74-8bd6-cf046ee9b44b None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:15.255510 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d7b34d-00ae-4f74-8bd6-cf046ee9b44b None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:15.260134 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847c2bba-f72a-4662-a8f8-a4c21bd72619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:15.260718 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847c2bba-f72a-4662-a8f8-a4c21bd72619 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:15.261000 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847c2bba-f72a-4662-a8f8-a4c21bd72619 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:15.261276 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847c2bba-f72a-4662-a8f8-a4c21bd72619 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:15.261947 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847c2bba-f72a-4662-a8f8-a4c21bd72619 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:15.263833 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847c2bba-f72a-4662-a8f8-a4c21bd72619 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:15.274631 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:39:15.294781 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1259/2492] 10.222.0.102 () {60 vars in 1158 bytes} [Thu Sep 1 17:39:15 2022] POST /identity/v3/projects => generated 404 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:39:15.296768 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9169c6-c037-470e-958a-583da7f74ebe admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:15.307434 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9169c6-c037-470e-958a-583da7f74ebe admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:15.365465 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288e3b92-2a6c-411c-a12b-71c0c81bf948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:15.366061 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288e3b92-2a6c-411c-a12b-71c0c81bf948 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:15.366336 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288e3b92-2a6c-411c-a12b-71c0c81bf948 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:15.366606 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288e3b92-2a6c-411c-a12b-71c0c81bf948 admin admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:15.367236 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288e3b92-2a6c-411c-a12b-71c0c81bf948 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:15.369132 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288e3b92-2a6c-411c-a12b-71c0c81bf948 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:15.390471 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1234/2493] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:39:15 2022] POST /identity/v3/users => generated 303 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:39:15.394713 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847c2bba-f72a-4662-a8f8-a4c21bd72619 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:15.405116 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847c2bba-f72a-4662-a8f8-a4c21bd72619 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:15.409846 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b52a24f-2fa6-455a-9500-ccd0c5e68adf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:15.410445 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b52a24f-2fa6-455a-9500-ccd0c5e68adf admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:15.411234 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b52a24f-2fa6-455a-9500-ccd0c5e68adf admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:15.411516 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b52a24f-2fa6-455a-9500-ccd0c5e68adf admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:15.412155 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b52a24f-2fa6-455a-9500-ccd0c5e68adf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:15.413776 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b52a24f-2fa6-455a-9500-ccd0c5e68adf admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:15.423643 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1260/2494] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:39:15 2022] GET /identity/v3/roles => generated 1427 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:39:15.428161 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288e3b92-2a6c-411c-a12b-71c0c81bf948 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:15.438067 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288e3b92-2a6c-411c-a12b-71c0c81bf948 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:15.442739 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767915d6-9218-4716-ae98-68df15276d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:15.443320 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767915d6-9218-4716-ae98-68df15276d7a admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:15.443600 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767915d6-9218-4716-ae98-68df15276d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:15.443873 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767915d6-9218-4716-ae98-68df15276d7a admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:15.444509 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767915d6-9218-4716-ae98-68df15276d7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:15.446122 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767915d6-9218-4716-ae98-68df15276d7a admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:15.456148 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1235/2495] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:39:15 2022] GET /identity/v3/roles => generated 1427 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:39:15.460819 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b52a24f-2fa6-455a-9500-ccd0c5e68adf admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:15.470776 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b52a24f-2fa6-455a-9500-ccd0c5e68adf admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:15.476060 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebb2615-4ac9-45e6-adfa-27dbdceb7c67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:15.476668 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebb2615-4ac9-45e6-adfa-27dbdceb7c67 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:15.476929 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebb2615-4ac9-45e6-adfa-27dbdceb7c67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:15.477212 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebb2615-4ac9-45e6-adfa-27dbdceb7c67 admin admin] PATH_INFO: `/v3/projects/6976f60656804ff1a7b8f11a3c2aa836/users/398ef1c60eae4b40b649c64fd2848176/roles/b75cb360925e4ba5a65f14965c103ffd` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:15.492405 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebb2615-4ac9-45e6-adfa-27dbdceb7c67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6976f60656804ff1a7b8f11a3c2aa836, user_id=398ef1c60eae4b40b649c64fd2848176, role_id=b75cb360925e4ba5a65f14965c103ffd)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:15.495107 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebb2615-4ac9-45e6-adfa-27dbdceb7c67 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:15.507801 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1261/2496] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:39:15 2022] PUT /identity/v3/projects/6976f60656804ff1a7b8f11a3c2aa836/users/398ef1c60eae4b40b649c64fd2848176/roles/b75cb360925e4ba5a65f14965c103ffd => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:39:15.518053 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2d3b70-2954-441b-bc65-b3bb449ecb4d None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:15.518339 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2d3b70-2954-441b-bc65-b3bb449ecb4d None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:15.518616 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2d3b70-2954-441b-bc65-b3bb449ecb4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:15.546358 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-4d2d3b70-2954-441b-bc65-b3bb449ecb4d None None] MFA Rules not processed for user `398ef1c60eae4b40b649c64fd2848176`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:39:15.547566 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-4d2d3b70-2954-441b-bc65-b3bb449ecb4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:39:15.566841 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:39:15.612607 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1236/2497] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:39:15 2022] POST /identity/v3/auth/tokens => generated 3290 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:39:15.617177 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebb2615-4ac9-45e6-adfa-27dbdceb7c67 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:15.677598 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebb2615-4ac9-45e6-adfa-27dbdceb7c67 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:15.682152 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547fd83c-a883-4a8b-9170-8bc4a54be75c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:15.682742 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547fd83c-a883-4a8b-9170-8bc4a54be75c admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:15.683018 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547fd83c-a883-4a8b-9170-8bc4a54be75c admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:15.683286 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547fd83c-a883-4a8b-9170-8bc4a54be75c admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:15.683899 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547fd83c-a883-4a8b-9170-8bc4a54be75c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:15.685817 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547fd83c-a883-4a8b-9170-8bc4a54be75c admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:15.712786 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1262/2498] 10.222.0.102 () {60 vars in 1158 bytes} [Thu Sep 1 17:39:15 2022] POST /identity/v3/projects => generated 406 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:39:15.716810 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2d3b70-2954-441b-bc65-b3bb449ecb4d None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:15.727989 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2d3b70-2954-441b-bc65-b3bb449ecb4d None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:15.782849 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b11fdc-a0ba-49bc-88c4-eedb57ca0778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:15.783451 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b11fdc-a0ba-49bc-88c4-eedb57ca0778 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:15.783734 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b11fdc-a0ba-49bc-88c4-eedb57ca0778 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:15.784038 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b11fdc-a0ba-49bc-88c4-eedb57ca0778 admin admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:15.784696 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b11fdc-a0ba-49bc-88c4-eedb57ca0778 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:15.786762 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b11fdc-a0ba-49bc-88c4-eedb57ca0778 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:15.807148 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1237/2499] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:39:15 2022] POST /identity/v3/users => generated 310 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:39:15.811157 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547fd83c-a883-4a8b-9170-8bc4a54be75c admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:15.822337 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547fd83c-a883-4a8b-9170-8bc4a54be75c admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:15.827007 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f3cc0a-68ca-4de4-92a3-bdcaab156257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:15.827597 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f3cc0a-68ca-4de4-92a3-bdcaab156257 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:15.828140 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f3cc0a-68ca-4de4-92a3-bdcaab156257 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:15.828415 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f3cc0a-68ca-4de4-92a3-bdcaab156257 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:15.829044 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f3cc0a-68ca-4de4-92a3-bdcaab156257 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:15.831004 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f3cc0a-68ca-4de4-92a3-bdcaab156257 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:15.841403 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1263/2500] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:39:15 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:39:15.845472 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b11fdc-a0ba-49bc-88c4-eedb57ca0778 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:15.856097 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b11fdc-a0ba-49bc-88c4-eedb57ca0778 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:15.860713 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b3d55a-e6be-45fa-b39d-c50e0502e879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:15.861382 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b3d55a-e6be-45fa-b39d-c50e0502e879 admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:15.861666 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b3d55a-e6be-45fa-b39d-c50e0502e879 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:15.861955 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b3d55a-e6be-45fa-b39d-c50e0502e879 admin admin] PATH_INFO: `/v3/projects/c6110d907a8c4a79aff445a876b1eaa3/users/ca20d3842248424a9cd0017aae1a467e/roles/36577a6f46d642d4b81805ede9872490` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:15.876893 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b3d55a-e6be-45fa-b39d-c50e0502e879 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6110d907a8c4a79aff445a876b1eaa3, user_id=ca20d3842248424a9cd0017aae1a467e, role_id=36577a6f46d642d4b81805ede9872490)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:15.879602 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b3d55a-e6be-45fa-b39d-c50e0502e879 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:15.891641 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1238/2501] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:39:15 2022] PUT /identity/v3/projects/c6110d907a8c4a79aff445a876b1eaa3/users/ca20d3842248424a9cd0017aae1a467e/roles/36577a6f46d642d4b81805ede9872490 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:39:15.895612 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f3cc0a-68ca-4de4-92a3-bdcaab156257 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:15.905565 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f3cc0a-68ca-4de4-92a3-bdcaab156257 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:15.961583 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c579848-b006-4d91-a4eb-a8e53bd24700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:15.962179 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c579848-b006-4d91-a4eb-a8e53bd24700 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:15.962493 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c579848-b006-4d91-a4eb-a8e53bd24700 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:15.962767 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c579848-b006-4d91-a4eb-a8e53bd24700 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:15.963390 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c579848-b006-4d91-a4eb-a8e53bd24700 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:15.964977 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c579848-b006-4d91-a4eb-a8e53bd24700 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:15.974934 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1264/2502] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:39:15 2022] GET /identity/v3/roles => generated 1427 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:39:15.979323 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b3d55a-e6be-45fa-b39d-c50e0502e879 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:15.990192 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b3d55a-e6be-45fa-b39d-c50e0502e879 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:15.994822 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c2f448-888f-4be8-9a41-52f179e42403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:15.995451 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c2f448-888f-4be8-9a41-52f179e42403 admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:15.995726 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c2f448-888f-4be8-9a41-52f179e42403 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:15.995993 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c2f448-888f-4be8-9a41-52f179e42403 admin admin] PATH_INFO: `/v3/projects/c6110d907a8c4a79aff445a876b1eaa3/users/ca20d3842248424a9cd0017aae1a467e/roles/36577a6f46d642d4b81805ede9872490` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:15.997916 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c2f448-888f-4be8-9a41-52f179e42403 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6110d907a8c4a79aff445a876b1eaa3, user_id=ca20d3842248424a9cd0017aae1a467e, role_id=36577a6f46d642d4b81805ede9872490)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:16.000515 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c2f448-888f-4be8-9a41-52f179e42403 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:16.010584 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1239/2503] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:39:15 2022] PUT /identity/v3/projects/c6110d907a8c4a79aff445a876b1eaa3/users/ca20d3842248424a9cd0017aae1a467e/roles/36577a6f46d642d4b81805ede9872490 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:39:16.020450 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e3e7f2-0f00-4d68-b5a4-54c703bd3a1d None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:16.020737 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e3e7f2-0f00-4d68-b5a4-54c703bd3a1d None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:16.021012 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e3e7f2-0f00-4d68-b5a4-54c703bd3a1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:16.048862 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-29e3e7f2-0f00-4d68-b5a4-54c703bd3a1d None None] MFA Rules not processed for user `ca20d3842248424a9cd0017aae1a467e`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:39:16.050102 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-29e3e7f2-0f00-4d68-b5a4-54c703bd3a1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:39:16.062804 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:39:16.119816 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1265/2504] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:39:16 2022] POST /identity/v3/auth/tokens => generated 3359 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:39:16.130461 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c2f448-888f-4be8-9a41-52f179e42403 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:16.132136 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-31c2f448-888f-4be8-9a41-52f179e42403 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:39:16.188453 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c2f448-888f-4be8-9a41-52f179e42403 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:16.192011 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee5d1a8-d295-4f39-9359-a86a585fceb1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '523ed589872a45a09b6a7675a7fb0b75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:16.192594 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee5d1a8-d295-4f39-9359-a86a585fceb1 service cinder] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:16.192870 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee5d1a8-d295-4f39-9359-a86a585fceb1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:16.193139 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee5d1a8-d295-4f39-9359-a86a585fceb1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:16.201762 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee5d1a8-d295-4f39-9359-a86a585fceb1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:16.203415 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee5d1a8-d295-4f39-9359-a86a585fceb1 service cinder] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:16.213009 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1240/2505] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:39:16 2022] GET /identity/v3/auth/tokens => generated 3359 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:39:16.365987 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e3e7f2-0f00-4d68-b5a4-54c703bd3a1d None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:16.426759 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e3e7f2-0f00-4d68-b5a4-54c703bd3a1d None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:16.431204 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb69e73-758f-49a7-a2ae-7023289dd436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:16.431826 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb69e73-758f-49a7-a2ae-7023289dd436 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:16.432107 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb69e73-758f-49a7-a2ae-7023289dd436 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:16.432381 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb69e73-758f-49a7-a2ae-7023289dd436 admin admin] PATH_INFO: `/v3/users/398ef1c60eae4b40b649c64fd2848176` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:16.433523 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb69e73-758f-49a7-a2ae-7023289dd436 admin admin] RBAC: Authorizing `identity:delete_user(user_id=398ef1c60eae4b40b649c64fd2848176)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:16.435445 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb69e73-758f-49a7-a2ae-7023289dd436 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:16.483008 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-bfb69e73-758f-49a7-a2ae-7023289dd436 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '398ef1c60eae4b40b649c64fd2848176'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:16.495175 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-bfb69e73-758f-49a7-a2ae-7023289dd436 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '398ef1c60eae4b40b649c64fd2848176'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:16.503010 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-bfb69e73-758f-49a7-a2ae-7023289dd436 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '398ef1c60eae4b40b649c64fd2848176'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:16.503704 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-bfb69e73-758f-49a7-a2ae-7023289dd436 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '398ef1c60eae4b40b649c64fd2848176'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:16.504278 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-bfb69e73-758f-49a7-a2ae-7023289dd436 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '398ef1c60eae4b40b649c64fd2848176'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:16.516400 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1266/2506] 10.222.0.102 () {58 vars in 1229 bytes} [Thu Sep 1 17:39:16 2022] DELETE /identity/v3/users/398ef1c60eae4b40b649c64fd2848176 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:39:16.521564 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f778b5-b018-432c-9307-eee5efb4d0c8 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:16.521853 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f778b5-b018-432c-9307-eee5efb4d0c8 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:16.522125 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f778b5-b018-432c-9307-eee5efb4d0c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:16.541248 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-53f778b5-b018-432c-9307-eee5efb4d0c8 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:39:16.542469 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-53f778b5-b018-432c-9307-eee5efb4d0c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:39:16.554334 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:39:16.601170 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1241/2507] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:39:16 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:39:16.610914 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb69e73-758f-49a7-a2ae-7023289dd436 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:16.612293 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-bfb69e73-758f-49a7-a2ae-7023289dd436 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:39:16.646562 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:39:16.671214 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb69e73-758f-49a7-a2ae-7023289dd436 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:16.674784 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d97503f-ca55-4e60-918d-95cabb8ec784 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:16.675402 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d97503f-ca55-4e60-918d-95cabb8ec784 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:16.675676 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d97503f-ca55-4e60-918d-95cabb8ec784 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:16.675947 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d97503f-ca55-4e60-918d-95cabb8ec784 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:16.684675 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d97503f-ca55-4e60-918d-95cabb8ec784 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:16.686189 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d97503f-ca55-4e60-918d-95cabb8ec784 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:16.695866 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1267/2508] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:39:16 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:39:16.958809 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f778b5-b018-432c-9307-eee5efb4d0c8 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:16.960303 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-53f778b5-b018-432c-9307-eee5efb4d0c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:39:16.972449 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f778b5-b018-432c-9307-eee5efb4d0c8 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:16.977331 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae61dc84-3345-4a60-bf6c-f4963e8dcb9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:16.978199 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae61dc84-3345-4a60-bf6c-f4963e8dcb9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:16.978561 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae61dc84-3345-4a60-bf6c-f4963e8dcb9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:16.978896 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae61dc84-3345-4a60-bf6c-f4963e8dcb9a admin admin] PATH_INFO: `/v3/projects/6976f60656804ff1a7b8f11a3c2aa836` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:16.980819 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae61dc84-3345-4a60-bf6c-f4963e8dcb9a admin admin] RBAC: Authorizing `identity:delete_project(project_id=6976f60656804ff1a7b8f11a3c2aa836)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:16.983165 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae61dc84-3345-4a60-bf6c-f4963e8dcb9a admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:17.030506 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-ae61dc84-3345-4a60-bf6c-f4963e8dcb9a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6976f60656804ff1a7b8f11a3c2aa836', 'request_id': 'req-ae61dc84-3345-4a60-bf6c-f4963e8dcb9a'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:17.036033 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-ae61dc84-3345-4a60-bf6c-f4963e8dcb9a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6976f60656804ff1a7b8f11a3c2aa836', 'request_id': 'req-ae61dc84-3345-4a60-bf6c-f4963e8dcb9a'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:17.043629 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-ae61dc84-3345-4a60-bf6c-f4963e8dcb9a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6976f60656804ff1a7b8f11a3c2aa836', 'request_id': 'req-ae61dc84-3345-4a60-bf6c-f4963e8dcb9a'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:17.054984 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-ae61dc84-3345-4a60-bf6c-f4963e8dcb9a admin admin] The token cache is being invalidate because project 6976f60656804ff1a7b8f11a3c2aa836 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:39:17.055273 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-ae61dc84-3345-4a60-bf6c-f4963e8dcb9a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:17.055929 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-ae61dc84-3345-4a60-bf6c-f4963e8dcb9a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:17.057396 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1242/2509] 10.222.0.102 () {58 vars in 1238 bytes} [Thu Sep 1 17:39:16 2022] DELETE /identity/v3/projects/6976f60656804ff1a7b8f11a3c2aa836 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:39:17.061437 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d97503f-ca55-4e60-918d-95cabb8ec784 service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:17.071560 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d97503f-ca55-4e60-918d-95cabb8ec784 service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:17.126757 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fac7b6-e79b-450c-bdfc-b588f4802c4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:17.127379 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fac7b6-e79b-450c-bdfc-b588f4802c4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:17.127660 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fac7b6-e79b-450c-bdfc-b588f4802c4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:17.127934 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fac7b6-e79b-450c-bdfc-b588f4802c4d admin admin] PATH_INFO: `/v3/users/ca20d3842248424a9cd0017aae1a467e` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:17.129032 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fac7b6-e79b-450c-bdfc-b588f4802c4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca20d3842248424a9cd0017aae1a467e)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:17.131009 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fac7b6-e79b-450c-bdfc-b588f4802c4d admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:17.176715 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-87fac7b6-e79b-450c-bdfc-b588f4802c4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca20d3842248424a9cd0017aae1a467e'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:17.188351 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-87fac7b6-e79b-450c-bdfc-b588f4802c4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca20d3842248424a9cd0017aae1a467e'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:17.195326 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-87fac7b6-e79b-450c-bdfc-b588f4802c4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca20d3842248424a9cd0017aae1a467e'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:17.196024 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-87fac7b6-e79b-450c-bdfc-b588f4802c4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca20d3842248424a9cd0017aae1a467e'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:17.196644 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-87fac7b6-e79b-450c-bdfc-b588f4802c4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca20d3842248424a9cd0017aae1a467e'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:17.209552 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1268/2510] 10.222.0.102 () {58 vars in 1229 bytes} [Thu Sep 1 17:39:17 2022] DELETE /identity/v3/users/ca20d3842248424a9cd0017aae1a467e => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:39:17.482450 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae61dc84-3345-4a60-bf6c-f4963e8dcb9a admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:17.483938 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-ae61dc84-3345-4a60-bf6c-f4963e8dcb9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:39:17.498560 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae61dc84-3345-4a60-bf6c-f4963e8dcb9a admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:17.557724 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f124da-c381-40e1-a2f4-a9f1ed10695e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:17.558350 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f124da-c381-40e1-a2f4-a9f1ed10695e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:17.558629 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f124da-c381-40e1-a2f4-a9f1ed10695e admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:17.558900 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f124da-c381-40e1-a2f4-a9f1ed10695e admin admin] PATH_INFO: `/v3/projects/c6110d907a8c4a79aff445a876b1eaa3` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:17.560016 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f124da-c381-40e1-a2f4-a9f1ed10695e admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6110d907a8c4a79aff445a876b1eaa3)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:17.562050 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f124da-c381-40e1-a2f4-a9f1ed10695e admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:17.603742 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:39:17.607823 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-d5f124da-c381-40e1-a2f4-a9f1ed10695e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6110d907a8c4a79aff445a876b1eaa3', 'request_id': 'req-d5f124da-c381-40e1-a2f4-a9f1ed10695e'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:17.614013 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-d5f124da-c381-40e1-a2f4-a9f1ed10695e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6110d907a8c4a79aff445a876b1eaa3', 'request_id': 'req-d5f124da-c381-40e1-a2f4-a9f1ed10695e'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:17.621196 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-d5f124da-c381-40e1-a2f4-a9f1ed10695e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6110d907a8c4a79aff445a876b1eaa3', 'request_id': 'req-d5f124da-c381-40e1-a2f4-a9f1ed10695e'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:17.632508 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-d5f124da-c381-40e1-a2f4-a9f1ed10695e admin admin] The token cache is being invalidate because project c6110d907a8c4a79aff445a876b1eaa3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:39:17.632792 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-d5f124da-c381-40e1-a2f4-a9f1ed10695e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:17.633474 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-d5f124da-c381-40e1-a2f4-a9f1ed10695e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:17.634935 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1243/2511] 10.222.0.102 () {58 vars in 1238 bytes} [Thu Sep 1 17:39:17 2022] DELETE /identity/v3/projects/c6110d907a8c4a79aff445a876b1eaa3 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:39:17.639954 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759eca80-28fd-465b-8ec3-0e5efd54359d None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:17.640248 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759eca80-28fd-465b-8ec3-0e5efd54359d None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:17.640528 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759eca80-28fd-465b-8ec3-0e5efd54359d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:17.660249 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-759eca80-28fd-465b-8ec3-0e5efd54359d None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:39:17.661550 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-759eca80-28fd-465b-8ec3-0e5efd54359d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:39:17.709405 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:39:17.723019 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1269/2512] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:39:17 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:39:17.732589 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650d5811-7a16-4d85-a6f8-67b1c147da00 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:17.732872 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650d5811-7a16-4d85-a6f8-67b1c147da00 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:17.733141 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650d5811-7a16-4d85-a6f8-67b1c147da00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:17.753792 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-650d5811-7a16-4d85-a6f8-67b1c147da00 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:39:17.755621 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-650d5811-7a16-4d85-a6f8-67b1c147da00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:39:17.763460 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1244/2513] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:39:17 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:39:17.767625 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759eca80-28fd-465b-8ec3-0e5efd54359d None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:17.777371 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759eca80-28fd-465b-8ec3-0e5efd54359d None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:17.781893 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71343974-a970-4eb9-9fb5-32b9d9c896aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:17.782481 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71343974-a970-4eb9-9fb5-32b9d9c896aa admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:17.782759 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71343974-a970-4eb9-9fb5-32b9d9c896aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:17.783028 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71343974-a970-4eb9-9fb5-32b9d9c896aa admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:17.783663 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71343974-a970-4eb9-9fb5-32b9d9c896aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:39:17.783962 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71343974-a970-4eb9-9fb5-32b9d9c896aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:17.785568 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71343974-a970-4eb9-9fb5-32b9d9c896aa admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:17.797180 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1270/2514] 10.222.0.102 () {58 vars in 1158 bytes} [Thu Sep 1 17:39:17 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:39:17.802022 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650d5811-7a16-4d85-a6f8-67b1c147da00 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:17.811386 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650d5811-7a16-4d85-a6f8-67b1c147da00 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:17.816044 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4467af8b-0e22-4217-9daf-98e55f29fea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:17.816635 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4467af8b-0e22-4217-9daf-98e55f29fea4 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:17.817666 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4467af8b-0e22-4217-9daf-98e55f29fea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:17.817968 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4467af8b-0e22-4217-9daf-98e55f29fea4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:17.818598 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4467af8b-0e22-4217-9daf-98e55f29fea4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:17.820559 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4467af8b-0e22-4217-9daf-98e55f29fea4 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:17.846879 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1245/2515] 10.222.0.102 () {60 vars in 1158 bytes} [Thu Sep 1 17:39:17 2022] POST /identity/v3/projects => generated 382 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:39:17.851172 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71343974-a970-4eb9-9fb5-32b9d9c896aa admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:17.861097 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71343974-a970-4eb9-9fb5-32b9d9c896aa admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:17.916734 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbb1227-4a50-4c22-9ff4-95fbd2612113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:17.917352 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbb1227-4a50-4c22-9ff4-95fbd2612113 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:17.917635 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbb1227-4a50-4c22-9ff4-95fbd2612113 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:17.917903 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbb1227-4a50-4c22-9ff4-95fbd2612113 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:17.918527 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbb1227-4a50-4c22-9ff4-95fbd2612113 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:17.920472 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbb1227-4a50-4c22-9ff4-95fbd2612113 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:17.941717 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1271/2516] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:39:17 2022] POST /identity/v3/users => generated 292 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:39:17.946766 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4467af8b-0e22-4217-9daf-98e55f29fea4 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:17.957184 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4467af8b-0e22-4217-9daf-98e55f29fea4 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:17.961853 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13188da-4190-48fb-a32b-551477ce1d61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:17.962414 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13188da-4190-48fb-a32b-551477ce1d61 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:17.962697 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13188da-4190-48fb-a32b-551477ce1d61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:17.962965 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13188da-4190-48fb-a32b-551477ce1d61 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:17.963589 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13188da-4190-48fb-a32b-551477ce1d61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:17.965147 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13188da-4190-48fb-a32b-551477ce1d61 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:17.975410 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1246/2517] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:39:17 2022] GET /identity/v3/roles => generated 1427 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:39:17.979337 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbb1227-4a50-4c22-9ff4-95fbd2612113 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:17.989535 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbb1227-4a50-4c22-9ff4-95fbd2612113 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:17.994391 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e9537c-4591-4788-9827-9f1ce5e8fd5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:17.994981 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e9537c-4591-4788-9827-9f1ce5e8fd5c admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:17.995302 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e9537c-4591-4788-9827-9f1ce5e8fd5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:17.995559 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e9537c-4591-4788-9827-9f1ce5e8fd5c admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:17.996193 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e9537c-4591-4788-9827-9f1ce5e8fd5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:17.997866 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e9537c-4591-4788-9827-9f1ce5e8fd5c admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:18.008830 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1272/2518] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:39:17 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:39:18.012678 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13188da-4190-48fb-a32b-551477ce1d61 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:18.022386 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13188da-4190-48fb-a32b-551477ce1d61 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:18.026830 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fcc70c-3e2b-43c7-ae62-cd573c1b6072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:18.027451 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fcc70c-3e2b-43c7-ae62-cd573c1b6072 admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:18.027725 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fcc70c-3e2b-43c7-ae62-cd573c1b6072 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:18.027990 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fcc70c-3e2b-43c7-ae62-cd573c1b6072 admin admin] PATH_INFO: `/v3/projects/41814427a55743a58f4c365124644892/users/60f1553912d94ded99c912c2fde8d092/roles/b75cb360925e4ba5a65f14965c103ffd` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:18.042425 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fcc70c-3e2b-43c7-ae62-cd573c1b6072 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41814427a55743a58f4c365124644892, user_id=60f1553912d94ded99c912c2fde8d092, role_id=b75cb360925e4ba5a65f14965c103ffd)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:18.045077 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fcc70c-3e2b-43c7-ae62-cd573c1b6072 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:18.058182 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1247/2519] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:39:18 2022] PUT /identity/v3/projects/41814427a55743a58f4c365124644892/users/60f1553912d94ded99c912c2fde8d092/roles/b75cb360925e4ba5a65f14965c103ffd => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:39:18.068520 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46e1474-d763-4086-969f-9870e87dc02f None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:18.068745 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46e1474-d763-4086-969f-9870e87dc02f None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:18.069076 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46e1474-d763-4086-969f-9870e87dc02f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:18.106247 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-c46e1474-d763-4086-969f-9870e87dc02f None None] MFA Rules not processed for user `60f1553912d94ded99c912c2fde8d092`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:39:18.107756 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-c46e1474-d763-4086-969f-9870e87dc02f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:39:18.172899 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1273/2520] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:39:18 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:39:18.178099 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fcc70c-3e2b-43c7-ae62-cd573c1b6072 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:18.238405 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fcc70c-3e2b-43c7-ae62-cd573c1b6072 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:18.243595 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea29d0dc-5515-4024-af40-67ffeec564c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:18.244209 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea29d0dc-5515-4024-af40-67ffeec564c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:18.244488 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea29d0dc-5515-4024-af40-67ffeec564c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:18.244759 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea29d0dc-5515-4024-af40-67ffeec564c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:18.245420 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea29d0dc-5515-4024-af40-67ffeec564c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:18.247307 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea29d0dc-5515-4024-af40-67ffeec564c0 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:18.272716 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1248/2521] 10.222.0.102 () {60 vars in 1158 bytes} [Thu Sep 1 17:39:18 2022] POST /identity/v3/projects => generated 382 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:39:18.276592 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46e1474-d763-4086-969f-9870e87dc02f None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:18.287873 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46e1474-d763-4086-969f-9870e87dc02f None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:18.332273 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:39:18.344353 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66acdd37-d0d2-44c0-9c85-b3f1f5ea65b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:18.344974 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66acdd37-d0d2-44c0-9c85-b3f1f5ea65b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:18.345251 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66acdd37-d0d2-44c0-9c85-b3f1f5ea65b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:18.345559 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66acdd37-d0d2-44c0-9c85-b3f1f5ea65b1 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:18.346183 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66acdd37-d0d2-44c0-9c85-b3f1f5ea65b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:18.348590 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66acdd37-d0d2-44c0-9c85-b3f1f5ea65b1 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:18.369451 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1274/2522] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:39:18 2022] POST /identity/v3/users => generated 298 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:39:18.373651 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea29d0dc-5515-4024-af40-67ffeec564c0 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:18.383750 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea29d0dc-5515-4024-af40-67ffeec564c0 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:18.388374 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e78eaa-ad1f-464d-afaa-253cbd87777c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:18.388964 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e78eaa-ad1f-464d-afaa-253cbd87777c admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:18.389245 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e78eaa-ad1f-464d-afaa-253cbd87777c admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:18.389547 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e78eaa-ad1f-464d-afaa-253cbd87777c admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:18.390177 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e78eaa-ad1f-464d-afaa-253cbd87777c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:18.391763 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e78eaa-ad1f-464d-afaa-253cbd87777c admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:18.402345 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1249/2523] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:39:18 2022] GET /identity/v3/roles => generated 1427 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:39:18.406302 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66acdd37-d0d2-44c0-9c85-b3f1f5ea65b1 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:18.416649 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66acdd37-d0d2-44c0-9c85-b3f1f5ea65b1 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:18.421378 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17874cbf-5789-43cc-990d-44847b84f42e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:18.422041 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17874cbf-5789-43cc-990d-44847b84f42e admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:18.422320 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17874cbf-5789-43cc-990d-44847b84f42e admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:18.422592 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17874cbf-5789-43cc-990d-44847b84f42e admin admin] PATH_INFO: `/v3/projects/78b12c4566a64aa59315b1e18fd69a16/users/7f22418a3a8a4044b7099a216e93fadb/roles/36577a6f46d642d4b81805ede9872490` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:18.437395 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17874cbf-5789-43cc-990d-44847b84f42e admin admin] RBAC: Authorizing `identity:create_grant(project_id=78b12c4566a64aa59315b1e18fd69a16, user_id=7f22418a3a8a4044b7099a216e93fadb, role_id=36577a6f46d642d4b81805ede9872490)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:18.440037 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17874cbf-5789-43cc-990d-44847b84f42e admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:18.453527 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1275/2524] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:39:18 2022] PUT /identity/v3/projects/78b12c4566a64aa59315b1e18fd69a16/users/7f22418a3a8a4044b7099a216e93fadb/roles/36577a6f46d642d4b81805ede9872490 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:39:18.458396 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e78eaa-ad1f-464d-afaa-253cbd87777c admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:18.470239 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e78eaa-ad1f-464d-afaa-253cbd87777c admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:18.524376 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:39:18.542345 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64fd2f4-6390-4557-964f-c48b5d1a0082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:18.542946 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64fd2f4-6390-4557-964f-c48b5d1a0082 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:18.543227 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64fd2f4-6390-4557-964f-c48b5d1a0082 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:18.543500 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64fd2f4-6390-4557-964f-c48b5d1a0082 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:18.544131 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64fd2f4-6390-4557-964f-c48b5d1a0082 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:18.545744 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64fd2f4-6390-4557-964f-c48b5d1a0082 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:18.555288 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1250/2525] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:39:18 2022] GET /identity/v3/roles => generated 1427 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:39:18.559752 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17874cbf-5789-43cc-990d-44847b84f42e admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:18.570141 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17874cbf-5789-43cc-990d-44847b84f42e admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:18.574756 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89af646f-daff-49c4-919e-1d23a988e017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:18.575394 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89af646f-daff-49c4-919e-1d23a988e017 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:18.575670 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89af646f-daff-49c4-919e-1d23a988e017 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:18.575940 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89af646f-daff-49c4-919e-1d23a988e017 admin admin] PATH_INFO: `/v3/projects/78b12c4566a64aa59315b1e18fd69a16/users/7f22418a3a8a4044b7099a216e93fadb/roles/36577a6f46d642d4b81805ede9872490` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:18.578488 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89af646f-daff-49c4-919e-1d23a988e017 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78b12c4566a64aa59315b1e18fd69a16, user_id=7f22418a3a8a4044b7099a216e93fadb, role_id=36577a6f46d642d4b81805ede9872490)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:18.581120 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89af646f-daff-49c4-919e-1d23a988e017 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:18.591809 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1276/2526] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:39:18 2022] PUT /identity/v3/projects/78b12c4566a64aa59315b1e18fd69a16/users/7f22418a3a8a4044b7099a216e93fadb/roles/36577a6f46d642d4b81805ede9872490 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:39:18.602909 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c4c763-ae0c-4bbe-8675-d89950614d97 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:18.603197 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c4c763-ae0c-4bbe-8675-d89950614d97 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:18.603501 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c4c763-ae0c-4bbe-8675-d89950614d97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:18.630575 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-a7c4c763-ae0c-4bbe-8675-d89950614d97 None None] MFA Rules not processed for user `7f22418a3a8a4044b7099a216e93fadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:39:18.631839 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-a7c4c763-ae0c-4bbe-8675-d89950614d97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:39:18.698229 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1251/2527] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:39:18 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:39:18.710184 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89af646f-daff-49c4-919e-1d23a988e017 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:18.711876 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-89af646f-daff-49c4-919e-1d23a988e017 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:39:18.771257 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89af646f-daff-49c4-919e-1d23a988e017 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:18.783858 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9471f7cc-304e-4cb0-af1b-f2f7858d2f53 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '523ed589872a45a09b6a7675a7fb0b75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:18.784482 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9471f7cc-304e-4cb0-af1b-f2f7858d2f53 service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:18.784760 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9471f7cc-304e-4cb0-af1b-f2f7858d2f53 service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:18.785029 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9471f7cc-304e-4cb0-af1b-f2f7858d2f53 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:18.795382 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9471f7cc-304e-4cb0-af1b-f2f7858d2f53 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:18.796811 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9471f7cc-304e-4cb0-af1b-f2f7858d2f53 service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:18.806112 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1277/2528] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:39:18 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:39:18.881466 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c4c763-ae0c-4bbe-8675-d89950614d97 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:18.891133 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c4c763-ae0c-4bbe-8675-d89950614d97 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:18.894741 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc5137e-f30b-4c33-a2c3-a9c8b7b90100 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '523ed589872a45a09b6a7675a7fb0b75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:18.895896 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc5137e-f30b-4c33-a2c3-a9c8b7b90100 service cinder] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:18.896173 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc5137e-f30b-4c33-a2c3-a9c8b7b90100 service cinder] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:18.896441 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc5137e-f30b-4c33-a2c3-a9c8b7b90100 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:18.951065 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc5137e-f30b-4c33-a2c3-a9c8b7b90100 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:18.952748 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc5137e-f30b-4c33-a2c3-a9c8b7b90100 service cinder] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:18.962024 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1252/2529] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:39:18 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:39:28.802015 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=48,keystone:INSERT=5 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:39:28.957644 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=72 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:39:29.830872 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9471f7cc-304e-4cb0-af1b-f2f7858d2f53 service cinder] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:29.892494 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9471f7cc-304e-4cb0-af1b-f2f7858d2f53 service cinder] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:29.897081 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c1ae1c-088e-414b-a6eb-af496abbd372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:29.897739 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c1ae1c-088e-414b-a6eb-af496abbd372 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:29.898024 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c1ae1c-088e-414b-a6eb-af496abbd372 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:29.898300 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c1ae1c-088e-414b-a6eb-af496abbd372 admin admin] PATH_INFO: `/v3/users/60f1553912d94ded99c912c2fde8d092` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:29.899548 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c1ae1c-088e-414b-a6eb-af496abbd372 admin admin] RBAC: Authorizing `identity:delete_user(user_id=60f1553912d94ded99c912c2fde8d092)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:29.901503 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c1ae1c-088e-414b-a6eb-af496abbd372 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:29.946631 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-39c1ae1c-088e-414b-a6eb-af496abbd372 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60f1553912d94ded99c912c2fde8d092'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:29.958490 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-39c1ae1c-088e-414b-a6eb-af496abbd372 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60f1553912d94ded99c912c2fde8d092'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:29.965511 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-39c1ae1c-088e-414b-a6eb-af496abbd372 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60f1553912d94ded99c912c2fde8d092'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:29.966126 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-39c1ae1c-088e-414b-a6eb-af496abbd372 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60f1553912d94ded99c912c2fde8d092'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:29.966714 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-39c1ae1c-088e-414b-a6eb-af496abbd372 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60f1553912d94ded99c912c2fde8d092'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:29.979334 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1278/2530] 10.222.0.102 () {58 vars in 1229 bytes} [Thu Sep 1 17:39:29 2022] DELETE /identity/v3/users/60f1553912d94ded99c912c2fde8d092 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:39:29.983933 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95bd36e-c32f-4bd6-aa89-7899811f559a None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:29.984441 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95bd36e-c32f-4bd6-aa89-7899811f559a None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:29.984546 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95bd36e-c32f-4bd6-aa89-7899811f559a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:30.003728 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-c95bd36e-c32f-4bd6-aa89-7899811f559a None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:39:30.004924 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-c95bd36e-c32f-4bd6-aa89-7899811f559a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:39:30.064887 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1253/2531] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:39:29 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:39:30.074697 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c1ae1c-088e-414b-a6eb-af496abbd372 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:30.076103 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-39c1ae1c-088e-414b-a6eb-af496abbd372 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:39:30.151725 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c1ae1c-088e-414b-a6eb-af496abbd372 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:30.155341 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95684e0-26a0-4e2e-80aa-1381690c095c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:30.155932 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95684e0-26a0-4e2e-80aa-1381690c095c service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:30.156209 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95684e0-26a0-4e2e-80aa-1381690c095c service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:30.156480 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95684e0-26a0-4e2e-80aa-1381690c095c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:30.165703 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95684e0-26a0-4e2e-80aa-1381690c095c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:30.167116 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95684e0-26a0-4e2e-80aa-1381690c095c service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:30.176217 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1279/2532] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:39:30 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:39:30.434338 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95bd36e-c32f-4bd6-aa89-7899811f559a None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:30.435956 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-c95bd36e-c32f-4bd6-aa89-7899811f559a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:39:30.450659 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95bd36e-c32f-4bd6-aa89-7899811f559a None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:30.455744 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3004425c-e3eb-4838-9c2a-d9ee20edb2ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:30.456374 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3004425c-e3eb-4838-9c2a-d9ee20edb2ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:30.456690 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3004425c-e3eb-4838-9c2a-d9ee20edb2ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:30.456911 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3004425c-e3eb-4838-9c2a-d9ee20edb2ba admin admin] PATH_INFO: `/v3/projects/41814427a55743a58f4c365124644892` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:30.458156 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3004425c-e3eb-4838-9c2a-d9ee20edb2ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=41814427a55743a58f4c365124644892)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:30.460539 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3004425c-e3eb-4838-9c2a-d9ee20edb2ba admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:30.506895 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-3004425c-e3eb-4838-9c2a-d9ee20edb2ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41814427a55743a58f4c365124644892', 'request_id': 'req-3004425c-e3eb-4838-9c2a-d9ee20edb2ba'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:30.512601 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-3004425c-e3eb-4838-9c2a-d9ee20edb2ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41814427a55743a58f4c365124644892', 'request_id': 'req-3004425c-e3eb-4838-9c2a-d9ee20edb2ba'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:30.522093 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-3004425c-e3eb-4838-9c2a-d9ee20edb2ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41814427a55743a58f4c365124644892', 'request_id': 'req-3004425c-e3eb-4838-9c2a-d9ee20edb2ba'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:30.537997 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-3004425c-e3eb-4838-9c2a-d9ee20edb2ba admin admin] The token cache is being invalidate because project 41814427a55743a58f4c365124644892 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:39:30.538289 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-3004425c-e3eb-4838-9c2a-d9ee20edb2ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:30.539032 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-3004425c-e3eb-4838-9c2a-d9ee20edb2ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:30.540517 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1254/2533] 10.222.0.102 () {58 vars in 1238 bytes} [Thu Sep 1 17:39:30 2022] DELETE /identity/v3/projects/41814427a55743a58f4c365124644892 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:39:30.544492 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95684e0-26a0-4e2e-80aa-1381690c095c service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:30.549393 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:39:30.555733 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95684e0-26a0-4e2e-80aa-1381690c095c service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:30.613788 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42340ca-ed3a-461a-9bd1-7e017531990c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:30.614396 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42340ca-ed3a-461a-9bd1-7e017531990c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:30.614678 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42340ca-ed3a-461a-9bd1-7e017531990c admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:30.614944 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42340ca-ed3a-461a-9bd1-7e017531990c admin admin] PATH_INFO: `/v3/users/7f22418a3a8a4044b7099a216e93fadb` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:30.616035 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42340ca-ed3a-461a-9bd1-7e017531990c admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f22418a3a8a4044b7099a216e93fadb)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:30.617972 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42340ca-ed3a-461a-9bd1-7e017531990c admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:30.659920 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-e42340ca-ed3a-461a-9bd1-7e017531990c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f22418a3a8a4044b7099a216e93fadb'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:30.671972 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-e42340ca-ed3a-461a-9bd1-7e017531990c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f22418a3a8a4044b7099a216e93fadb'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:30.678571 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-e42340ca-ed3a-461a-9bd1-7e017531990c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f22418a3a8a4044b7099a216e93fadb'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:30.679317 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-e42340ca-ed3a-461a-9bd1-7e017531990c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f22418a3a8a4044b7099a216e93fadb'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:30.679933 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-e42340ca-ed3a-461a-9bd1-7e017531990c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f22418a3a8a4044b7099a216e93fadb'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:30.691790 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1280/2534] 10.222.0.102 () {58 vars in 1229 bytes} [Thu Sep 1 17:39:30 2022] DELETE /identity/v3/users/7f22418a3a8a4044b7099a216e93fadb => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:39:30.958160 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3004425c-e3eb-4838-9c2a-d9ee20edb2ba admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:30.960329 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-3004425c-e3eb-4838-9c2a-d9ee20edb2ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:39:30.973662 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3004425c-e3eb-4838-9c2a-d9ee20edb2ba admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:31.030915 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1d19e8-a3b2-46e9-9951-85b59a0da71a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:31.031564 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1d19e8-a3b2-46e9-9951-85b59a0da71a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:31.031807 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1d19e8-a3b2-46e9-9951-85b59a0da71a admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:31.032082 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1d19e8-a3b2-46e9-9951-85b59a0da71a admin admin] PATH_INFO: `/v3/projects/78b12c4566a64aa59315b1e18fd69a16` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:31.033199 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1d19e8-a3b2-46e9-9951-85b59a0da71a admin admin] RBAC: Authorizing `identity:delete_project(project_id=78b12c4566a64aa59315b1e18fd69a16)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:31.035146 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1d19e8-a3b2-46e9-9951-85b59a0da71a admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:31.064816 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:39:31.082700 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-ee1d19e8-a3b2-46e9-9951-85b59a0da71a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78b12c4566a64aa59315b1e18fd69a16', 'request_id': 'req-ee1d19e8-a3b2-46e9-9951-85b59a0da71a'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:31.089274 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-ee1d19e8-a3b2-46e9-9951-85b59a0da71a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78b12c4566a64aa59315b1e18fd69a16', 'request_id': 'req-ee1d19e8-a3b2-46e9-9951-85b59a0da71a'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:31.096405 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-ee1d19e8-a3b2-46e9-9951-85b59a0da71a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78b12c4566a64aa59315b1e18fd69a16', 'request_id': 'req-ee1d19e8-a3b2-46e9-9951-85b59a0da71a'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:31.107014 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-ee1d19e8-a3b2-46e9-9951-85b59a0da71a admin admin] The token cache is being invalidate because project 78b12c4566a64aa59315b1e18fd69a16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:39:31.107296 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-ee1d19e8-a3b2-46e9-9951-85b59a0da71a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:31.107942 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-ee1d19e8-a3b2-46e9-9951-85b59a0da71a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:31.109420 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1255/2535] 10.222.0.102 () {58 vars in 1238 bytes} [Thu Sep 1 17:39:30 2022] DELETE /identity/v3/projects/78b12c4566a64aa59315b1e18fd69a16 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:39:31.114317 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bebfa3-6c14-49ee-a252-8d67bee101ec None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:31.115333 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bebfa3-6c14-49ee-a252-8d67bee101ec None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:31.115450 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bebfa3-6c14-49ee-a252-8d67bee101ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:31.134921 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-59bebfa3-6c14-49ee-a252-8d67bee101ec None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:39:31.136117 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-59bebfa3-6c14-49ee-a252-8d67bee101ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:39:31.192646 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1281/2536] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:39:31 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:39:31.201033 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5691a5b4-bc41-4f0b-9794-c5f26e5a1862 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:31.201408 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5691a5b4-bc41-4f0b-9794-c5f26e5a1862 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:31.201687 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5691a5b4-bc41-4f0b-9794-c5f26e5a1862 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:31.221630 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-5691a5b4-bc41-4f0b-9794-c5f26e5a1862 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:39:31.222826 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-5691a5b4-bc41-4f0b-9794-c5f26e5a1862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:39:31.230827 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1256/2537] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:39:31 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:39:31.234951 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bebfa3-6c14-49ee-a252-8d67bee101ec None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:31.244900 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bebfa3-6c14-49ee-a252-8d67bee101ec None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:31.249493 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc78280-f37b-4312-9ff9-cb8ceefa6638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:31.250085 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc78280-f37b-4312-9ff9-cb8ceefa6638 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:31.250366 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc78280-f37b-4312-9ff9-cb8ceefa6638 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:31.250635 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc78280-f37b-4312-9ff9-cb8ceefa6638 admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:31.251273 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc78280-f37b-4312-9ff9-cb8ceefa6638 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:39:31.251575 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc78280-f37b-4312-9ff9-cb8ceefa6638 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:31.253152 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc78280-f37b-4312-9ff9-cb8ceefa6638 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:31.265061 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1282/2538] 10.222.0.102 () {58 vars in 1158 bytes} [Thu Sep 1 17:39:31 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:39:31.269127 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5691a5b4-bc41-4f0b-9794-c5f26e5a1862 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:31.278023 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5691a5b4-bc41-4f0b-9794-c5f26e5a1862 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:31.282611 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f33d556-e19c-4d7e-9ac0-f81367c4e544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:31.283188 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f33d556-e19c-4d7e-9ac0-f81367c4e544 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:31.283463 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f33d556-e19c-4d7e-9ac0-f81367c4e544 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:31.283733 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f33d556-e19c-4d7e-9ac0-f81367c4e544 admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:31.284364 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f33d556-e19c-4d7e-9ac0-f81367c4e544 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:31.286268 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f33d556-e19c-4d7e-9ac0-f81367c4e544 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:31.312931 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1257/2539] 10.222.0.102 () {60 vars in 1158 bytes} [Thu Sep 1 17:39:31 2022] POST /identity/v3/projects => generated 358 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:39:31.316976 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc78280-f37b-4312-9ff9-cb8ceefa6638 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:31.328873 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc78280-f37b-4312-9ff9-cb8ceefa6638 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:31.333455 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:39:31.385625 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933926f5-8936-4670-98ca-a725576e7c06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:31.386218 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933926f5-8936-4670-98ca-a725576e7c06 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:31.386495 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933926f5-8936-4670-98ca-a725576e7c06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:31.386763 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933926f5-8936-4670-98ca-a725576e7c06 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:31.387408 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933926f5-8936-4670-98ca-a725576e7c06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:31.389324 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933926f5-8936-4670-98ca-a725576e7c06 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:31.409758 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1283/2540] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:39:31 2022] POST /identity/v3/users => generated 280 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:39:31.414002 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f33d556-e19c-4d7e-9ac0-f81367c4e544 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:31.424224 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f33d556-e19c-4d7e-9ac0-f81367c4e544 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:31.429512 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79505bf2-ee03-4cd5-bdea-fc8981c3b210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:31.430098 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79505bf2-ee03-4cd5-bdea-fc8981c3b210 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:31.430375 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79505bf2-ee03-4cd5-bdea-fc8981c3b210 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:31.430644 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79505bf2-ee03-4cd5-bdea-fc8981c3b210 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:31.431275 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79505bf2-ee03-4cd5-bdea-fc8981c3b210 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:31.432843 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79505bf2-ee03-4cd5-bdea-fc8981c3b210 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:31.442620 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1258/2541] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:39:31 2022] GET /identity/v3/roles => generated 1427 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:39:31.446908 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933926f5-8936-4670-98ca-a725576e7c06 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:31.456816 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933926f5-8936-4670-98ca-a725576e7c06 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:31.461475 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2545d0c2-e538-4e54-93c7-6361971f9479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:31.462080 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2545d0c2-e538-4e54-93c7-6361971f9479 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:31.462358 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2545d0c2-e538-4e54-93c7-6361971f9479 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:31.462631 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2545d0c2-e538-4e54-93c7-6361971f9479 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:31.463256 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2545d0c2-e538-4e54-93c7-6361971f9479 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:31.464828 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2545d0c2-e538-4e54-93c7-6361971f9479 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:31.475540 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1284/2542] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:39:31 2022] GET /identity/v3/roles => generated 1427 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 17:39:31.479525 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79505bf2-ee03-4cd5-bdea-fc8981c3b210 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:31.489294 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79505bf2-ee03-4cd5-bdea-fc8981c3b210 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:31.494529 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63676f5-a3de-48fb-a990-fdfbba246bd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:31.495164 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63676f5-a3de-48fb-a990-fdfbba246bd9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:31.495444 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63676f5-a3de-48fb-a990-fdfbba246bd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:31.495718 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63676f5-a3de-48fb-a990-fdfbba246bd9 admin admin] PATH_INFO: `/v3/projects/a648f4766dd4425ca1efb706f9cacef6/users/73593e048db446629909c2a603b4416e/roles/b75cb360925e4ba5a65f14965c103ffd` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:31.509876 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63676f5-a3de-48fb-a990-fdfbba246bd9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a648f4766dd4425ca1efb706f9cacef6, user_id=73593e048db446629909c2a603b4416e, role_id=b75cb360925e4ba5a65f14965c103ffd)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:31.512503 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63676f5-a3de-48fb-a990-fdfbba246bd9 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:31.524991 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1259/2543] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:39:31 2022] PUT /identity/v3/projects/a648f4766dd4425ca1efb706f9cacef6/users/73593e048db446629909c2a603b4416e/roles/b75cb360925e4ba5a65f14965c103ffd => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:39:31.535115 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc03543-d757-4b0d-9e35-b43db39401f4 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:31.535655 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc03543-d757-4b0d-9e35-b43db39401f4 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:31.535933 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc03543-d757-4b0d-9e35-b43db39401f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:31.563806 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-5dc03543-d757-4b0d-9e35-b43db39401f4 None None] MFA Rules not processed for user `73593e048db446629909c2a603b4416e`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:39:31.565024 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-5dc03543-d757-4b0d-9e35-b43db39401f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:39:31.626475 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1285/2544] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:39:31 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:39:31.630970 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63676f5-a3de-48fb-a990-fdfbba246bd9 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:31.691725 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63676f5-a3de-48fb-a990-fdfbba246bd9 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:31.696343 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c317c7c3-b0b4-47d2-b8e5-76ab371587cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:31.696934 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c317c7c3-b0b4-47d2-b8e5-76ab371587cc admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:31.697217 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c317c7c3-b0b4-47d2-b8e5-76ab371587cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:31.697527 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c317c7c3-b0b4-47d2-b8e5-76ab371587cc admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:31.698148 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c317c7c3-b0b4-47d2-b8e5-76ab371587cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:31.700052 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c317c7c3-b0b4-47d2-b8e5-76ab371587cc admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:31.726660 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1260/2545] 10.222.0.102 () {60 vars in 1158 bytes} [Thu Sep 1 17:39:31 2022] POST /identity/v3/projects => generated 356 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:39:31.730869 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc03543-d757-4b0d-9e35-b43db39401f4 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:31.742043 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc03543-d757-4b0d-9e35-b43db39401f4 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:31.796536 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51027600-4dfa-4114-9169-d3b2b5c93d25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:31.797167 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51027600-4dfa-4114-9169-d3b2b5c93d25 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:31.797477 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51027600-4dfa-4114-9169-d3b2b5c93d25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:31.797753 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51027600-4dfa-4114-9169-d3b2b5c93d25 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:31.798387 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51027600-4dfa-4114-9169-d3b2b5c93d25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:31.800298 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51027600-4dfa-4114-9169-d3b2b5c93d25 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:31.823625 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1286/2546] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:39:31 2022] POST /identity/v3/users => generated 285 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:39:31.828292 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c317c7c3-b0b4-47d2-b8e5-76ab371587cc admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:31.838460 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c317c7c3-b0b4-47d2-b8e5-76ab371587cc admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:31.843070 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a360795-5fef-479f-bb8b-ca41ab983a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:31.843652 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a360795-5fef-479f-bb8b-ca41ab983a99 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:31.843935 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a360795-5fef-479f-bb8b-ca41ab983a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:31.844205 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a360795-5fef-479f-bb8b-ca41ab983a99 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:31.844838 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a360795-5fef-479f-bb8b-ca41ab983a99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:31.846796 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a360795-5fef-479f-bb8b-ca41ab983a99 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:31.858182 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1261/2547] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:39:31 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:39:31.862318 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51027600-4dfa-4114-9169-d3b2b5c93d25 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:31.872960 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51027600-4dfa-4114-9169-d3b2b5c93d25 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:31.877608 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c764bf8-f684-4d77-9070-37c1f5cd5551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:31.878185 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c764bf8-f684-4d77-9070-37c1f5cd5551 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:31.878488 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c764bf8-f684-4d77-9070-37c1f5cd5551 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:31.878746 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c764bf8-f684-4d77-9070-37c1f5cd5551 admin admin] PATH_INFO: `/v3/projects/9beeadad304f458e9f25aa02095f610e/users/a481d5064a904874ba2af8a6a984dfc0/roles/36577a6f46d642d4b81805ede9872490` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:31.880932 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:39:31.894811 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c764bf8-f684-4d77-9070-37c1f5cd5551 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9beeadad304f458e9f25aa02095f610e, user_id=a481d5064a904874ba2af8a6a984dfc0, role_id=36577a6f46d642d4b81805ede9872490)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:31.898042 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c764bf8-f684-4d77-9070-37c1f5cd5551 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:31.910734 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1287/2548] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:39:31 2022] PUT /identity/v3/projects/9beeadad304f458e9f25aa02095f610e/users/a481d5064a904874ba2af8a6a984dfc0/roles/36577a6f46d642d4b81805ede9872490 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:39:31.915227 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a360795-5fef-479f-bb8b-ca41ab983a99 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:31.925094 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a360795-5fef-479f-bb8b-ca41ab983a99 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:31.945526 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:39:31.984152 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f949327-3fc2-4082-b2ab-7213eb2e0f1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:31.984747 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f949327-3fc2-4082-b2ab-7213eb2e0f1d admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:31.985027 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f949327-3fc2-4082-b2ab-7213eb2e0f1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:31.985326 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f949327-3fc2-4082-b2ab-7213eb2e0f1d admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:31.985958 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f949327-3fc2-4082-b2ab-7213eb2e0f1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:31.987536 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f949327-3fc2-4082-b2ab-7213eb2e0f1d admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:31.997559 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1262/2549] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:39:31 2022] GET /identity/v3/roles => generated 1427 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:39:32.001689 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c764bf8-f684-4d77-9070-37c1f5cd5551 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:32.012609 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c764bf8-f684-4d77-9070-37c1f5cd5551 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:32.017150 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892e7ab3-dc06-4b22-8027-a6c505505d84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:32.017852 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892e7ab3-dc06-4b22-8027-a6c505505d84 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:32.018134 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892e7ab3-dc06-4b22-8027-a6c505505d84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:32.018406 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892e7ab3-dc06-4b22-8027-a6c505505d84 admin admin] PATH_INFO: `/v3/projects/9beeadad304f458e9f25aa02095f610e/users/a481d5064a904874ba2af8a6a984dfc0/roles/36577a6f46d642d4b81805ede9872490` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:32.020415 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892e7ab3-dc06-4b22-8027-a6c505505d84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9beeadad304f458e9f25aa02095f610e, user_id=a481d5064a904874ba2af8a6a984dfc0, role_id=36577a6f46d642d4b81805ede9872490)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:32.023081 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892e7ab3-dc06-4b22-8027-a6c505505d84 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:32.033045 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1288/2550] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:39:32 2022] PUT /identity/v3/projects/9beeadad304f458e9f25aa02095f610e/users/a481d5064a904874ba2af8a6a984dfc0/roles/36577a6f46d642d4b81805ede9872490 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:39:32.043937 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6251b22-3b81-4f6c-afbd-0823d40a046e None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:32.044395 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6251b22-3b81-4f6c-afbd-0823d40a046e None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:32.044502 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6251b22-3b81-4f6c-afbd-0823d40a046e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:32.072900 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-d6251b22-3b81-4f6c-afbd-0823d40a046e None None] MFA Rules not processed for user `a481d5064a904874ba2af8a6a984dfc0`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:39:32.074140 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-d6251b22-3b81-4f6c-afbd-0823d40a046e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:39:32.140982 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1263/2551] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:39:32 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:39:32.152589 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892e7ab3-dc06-4b22-8027-a6c505505d84 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:32.154367 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-892e7ab3-dc06-4b22-8027-a6c505505d84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:39:32.233032 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892e7ab3-dc06-4b22-8027-a6c505505d84 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:32.236770 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c93863d-7daf-4d51-b3cf-4f580209aa78 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '523ed589872a45a09b6a7675a7fb0b75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:32.237391 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c93863d-7daf-4d51-b3cf-4f580209aa78 service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:32.237677 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c93863d-7daf-4d51-b3cf-4f580209aa78 service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:32.237952 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c93863d-7daf-4d51-b3cf-4f580209aa78 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:32.246748 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c93863d-7daf-4d51-b3cf-4f580209aa78 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:32.248178 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c93863d-7daf-4d51-b3cf-4f580209aa78 service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:32.258069 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1289/2552] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:39:32 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:39:32.336125 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6251b22-3b81-4f6c-afbd-0823d40a046e None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:32.345793 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6251b22-3b81-4f6c-afbd-0823d40a046e None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:32.349332 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047e3c53-9a4a-4805-b391-eb6bc06a1f4a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '523ed589872a45a09b6a7675a7fb0b75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:32.350438 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047e3c53-9a4a-4805-b391-eb6bc06a1f4a service cinder] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:32.350716 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047e3c53-9a4a-4805-b391-eb6bc06a1f4a service cinder] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:32.350986 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047e3c53-9a4a-4805-b391-eb6bc06a1f4a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:32.406751 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047e3c53-9a4a-4805-b391-eb6bc06a1f4a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:32.408107 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047e3c53-9a4a-4805-b391-eb6bc06a1f4a service cinder] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:32.417362 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1264/2553] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:39:32 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:39:42.253470 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=47,keystone:INSERT=2 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:39:42.413412 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=80 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:39:47.236343 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c93863d-7daf-4d51-b3cf-4f580209aa78 service cinder] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:47.305697 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c93863d-7daf-4d51-b3cf-4f580209aa78 service cinder] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:47.310532 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5d2c63-7063-47a7-b501-afa10a64321a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:47.311168 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5d2c63-7063-47a7-b501-afa10a64321a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:47.311451 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5d2c63-7063-47a7-b501-afa10a64321a admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:47.311835 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5d2c63-7063-47a7-b501-afa10a64321a admin admin] PATH_INFO: `/v3/users/73593e048db446629909c2a603b4416e` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:47.312840 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5d2c63-7063-47a7-b501-afa10a64321a admin admin] RBAC: Authorizing `identity:delete_user(user_id=73593e048db446629909c2a603b4416e)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:47.314801 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5d2c63-7063-47a7-b501-afa10a64321a admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:47.367333 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-5c5d2c63-7063-47a7-b501-afa10a64321a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73593e048db446629909c2a603b4416e'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:47.379699 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-5c5d2c63-7063-47a7-b501-afa10a64321a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73593e048db446629909c2a603b4416e'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:47.388062 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-5c5d2c63-7063-47a7-b501-afa10a64321a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73593e048db446629909c2a603b4416e'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:47.388813 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-5c5d2c63-7063-47a7-b501-afa10a64321a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73593e048db446629909c2a603b4416e'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:47.389445 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-5c5d2c63-7063-47a7-b501-afa10a64321a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73593e048db446629909c2a603b4416e'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:47.404385 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1290/2554] 10.222.0.102 () {58 vars in 1229 bytes} [Thu Sep 1 17:39:47 2022] DELETE /identity/v3/users/73593e048db446629909c2a603b4416e => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:39:47.408890 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b742c13d-4b94-4a38-8ec0-b9a6ca63de4d None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:47.409567 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b742c13d-4b94-4a38-8ec0-b9a6ca63de4d None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:47.409567 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b742c13d-4b94-4a38-8ec0-b9a6ca63de4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:47.430182 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-b742c13d-4b94-4a38-8ec0-b9a6ca63de4d None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:39:47.431641 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-b742c13d-4b94-4a38-8ec0-b9a6ca63de4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:39:47.492248 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1265/2555] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:39:47 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:39:47.501906 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5d2c63-7063-47a7-b501-afa10a64321a admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:47.503413 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-5c5d2c63-7063-47a7-b501-afa10a64321a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:39:47.560437 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5d2c63-7063-47a7-b501-afa10a64321a admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:47.564029 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f66299-13a1-4508-9255-f7baa14ac018 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:47.564622 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f66299-13a1-4508-9255-f7baa14ac018 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:47.564897 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f66299-13a1-4508-9255-f7baa14ac018 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:47.565166 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f66299-13a1-4508-9255-f7baa14ac018 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:47.574397 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f66299-13a1-4508-9255-f7baa14ac018 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:47.575811 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f66299-13a1-4508-9255-f7baa14ac018 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:47.585885 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1291/2556] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:39:47 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:39:47.847706 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b742c13d-4b94-4a38-8ec0-b9a6ca63de4d None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:47.849167 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-b742c13d-4b94-4a38-8ec0-b9a6ca63de4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:39:47.863268 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b742c13d-4b94-4a38-8ec0-b9a6ca63de4d None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:47.868669 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064ac937-6186-4738-adfb-e405c726b92e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:47.869491 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064ac937-6186-4738-adfb-e405c726b92e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:47.869812 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064ac937-6186-4738-adfb-e405c726b92e admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:47.870078 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064ac937-6186-4738-adfb-e405c726b92e admin admin] PATH_INFO: `/v3/projects/a648f4766dd4425ca1efb706f9cacef6` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:47.871215 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064ac937-6186-4738-adfb-e405c726b92e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a648f4766dd4425ca1efb706f9cacef6)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:47.873149 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064ac937-6186-4738-adfb-e405c726b92e admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:47.921239 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-064ac937-6186-4738-adfb-e405c726b92e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a648f4766dd4425ca1efb706f9cacef6', 'request_id': 'req-064ac937-6186-4738-adfb-e405c726b92e'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:47.927447 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-064ac937-6186-4738-adfb-e405c726b92e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a648f4766dd4425ca1efb706f9cacef6', 'request_id': 'req-064ac937-6186-4738-adfb-e405c726b92e'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:47.935748 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-064ac937-6186-4738-adfb-e405c726b92e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a648f4766dd4425ca1efb706f9cacef6', 'request_id': 'req-064ac937-6186-4738-adfb-e405c726b92e'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:47.949465 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-064ac937-6186-4738-adfb-e405c726b92e admin admin] The token cache is being invalidate because project a648f4766dd4425ca1efb706f9cacef6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:39:47.949759 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-064ac937-6186-4738-adfb-e405c726b92e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:47.950421 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-064ac937-6186-4738-adfb-e405c726b92e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:47.952159 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1266/2557] 10.222.0.102 () {58 vars in 1238 bytes} [Thu Sep 1 17:39:47 2022] DELETE /identity/v3/projects/a648f4766dd4425ca1efb706f9cacef6 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:39:47.956112 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f66299-13a1-4508-9255-f7baa14ac018 service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:47.967178 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f66299-13a1-4508-9255-f7baa14ac018 service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:47.985443 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:39:48.025862 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e2150a-f5da-4ec3-a795-55b71462694c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:48.026493 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e2150a-f5da-4ec3-a795-55b71462694c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:48.026774 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e2150a-f5da-4ec3-a795-55b71462694c admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:48.027046 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e2150a-f5da-4ec3-a795-55b71462694c admin admin] PATH_INFO: `/v3/users/a481d5064a904874ba2af8a6a984dfc0` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:48.028158 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e2150a-f5da-4ec3-a795-55b71462694c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a481d5064a904874ba2af8a6a984dfc0)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:48.030125 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e2150a-f5da-4ec3-a795-55b71462694c admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:48.074943 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-f6e2150a-f5da-4ec3-a795-55b71462694c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a481d5064a904874ba2af8a6a984dfc0'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:48.086845 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-f6e2150a-f5da-4ec3-a795-55b71462694c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a481d5064a904874ba2af8a6a984dfc0'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:48.093996 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-f6e2150a-f5da-4ec3-a795-55b71462694c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a481d5064a904874ba2af8a6a984dfc0'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:48.094776 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-f6e2150a-f5da-4ec3-a795-55b71462694c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a481d5064a904874ba2af8a6a984dfc0'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:48.095391 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-f6e2150a-f5da-4ec3-a795-55b71462694c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a481d5064a904874ba2af8a6a984dfc0'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:48.107961 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1292/2558] 10.222.0.102 () {58 vars in 1229 bytes} [Thu Sep 1 17:39:47 2022] DELETE /identity/v3/users/a481d5064a904874ba2af8a6a984dfc0 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:39:48.376947 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064ac937-6186-4738-adfb-e405c726b92e admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:48.378452 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-064ac937-6186-4738-adfb-e405c726b92e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:39:48.392499 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064ac937-6186-4738-adfb-e405c726b92e admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:48.451191 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44fb2d8-14a5-4b76-a77d-79e31eab2187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:48.451806 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44fb2d8-14a5-4b76-a77d-79e31eab2187 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:48.452085 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44fb2d8-14a5-4b76-a77d-79e31eab2187 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:48.452358 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44fb2d8-14a5-4b76-a77d-79e31eab2187 admin admin] PATH_INFO: `/v3/projects/9beeadad304f458e9f25aa02095f610e` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:48.453499 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44fb2d8-14a5-4b76-a77d-79e31eab2187 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9beeadad304f458e9f25aa02095f610e)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:48.455415 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44fb2d8-14a5-4b76-a77d-79e31eab2187 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:48.486599 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:39:48.506778 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-e44fb2d8-14a5-4b76-a77d-79e31eab2187 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9beeadad304f458e9f25aa02095f610e', 'request_id': 'req-e44fb2d8-14a5-4b76-a77d-79e31eab2187'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:48.513274 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-e44fb2d8-14a5-4b76-a77d-79e31eab2187 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9beeadad304f458e9f25aa02095f610e', 'request_id': 'req-e44fb2d8-14a5-4b76-a77d-79e31eab2187'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:48.521130 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-e44fb2d8-14a5-4b76-a77d-79e31eab2187 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9beeadad304f458e9f25aa02095f610e', 'request_id': 'req-e44fb2d8-14a5-4b76-a77d-79e31eab2187'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:48.532789 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-e44fb2d8-14a5-4b76-a77d-79e31eab2187 admin admin] The token cache is being invalidate because project 9beeadad304f458e9f25aa02095f610e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:39:48.533078 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-e44fb2d8-14a5-4b76-a77d-79e31eab2187 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:48.533792 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-e44fb2d8-14a5-4b76-a77d-79e31eab2187 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:48.535317 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1267/2559] 10.222.0.102 () {58 vars in 1238 bytes} [Thu Sep 1 17:39:48 2022] DELETE /identity/v3/projects/9beeadad304f458e9f25aa02095f610e => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:39:48.540120 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a259aba1-d90c-4591-926b-0eb2f40fac3d None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:48.540578 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a259aba1-d90c-4591-926b-0eb2f40fac3d None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:48.540690 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a259aba1-d90c-4591-926b-0eb2f40fac3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:48.560741 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-a259aba1-d90c-4591-926b-0eb2f40fac3d None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:39:48.561968 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-a259aba1-d90c-4591-926b-0eb2f40fac3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:39:48.624789 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1293/2560] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:39:48 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:39:48.633211 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8dc128-00a9-4abc-828f-0a83756f1122 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:48.633532 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8dc128-00a9-4abc-828f-0a83756f1122 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:48.633812 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8dc128-00a9-4abc-828f-0a83756f1122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:48.653962 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-fb8dc128-00a9-4abc-828f-0a83756f1122 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:39:48.655456 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-fb8dc128-00a9-4abc-828f-0a83756f1122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:39:48.663390 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1268/2561] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:39:48 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:39:48.667651 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a259aba1-d90c-4591-926b-0eb2f40fac3d None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:48.678718 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a259aba1-d90c-4591-926b-0eb2f40fac3d None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:48.683595 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cda465-751f-4df9-9436-27f6f4fe957c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:48.684196 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cda465-751f-4df9-9436-27f6f4fe957c admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:48.684478 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cda465-751f-4df9-9436-27f6f4fe957c admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:48.684752 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cda465-751f-4df9-9436-27f6f4fe957c admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:48.685433 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cda465-751f-4df9-9436-27f6f4fe957c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:39:48.685749 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cda465-751f-4df9-9436-27f6f4fe957c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:48.687355 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cda465-751f-4df9-9436-27f6f4fe957c admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:48.700209 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1294/2562] 10.222.0.102 () {58 vars in 1158 bytes} [Thu Sep 1 17:39:48 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:39:48.705270 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8dc128-00a9-4abc-828f-0a83756f1122 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:48.714635 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8dc128-00a9-4abc-828f-0a83756f1122 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:48.719277 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7d663c-8c9f-409c-b5b6-8d76c9bd572a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:48.719921 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7d663c-8c9f-409c-b5b6-8d76c9bd572a admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:48.720204 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7d663c-8c9f-409c-b5b6-8d76c9bd572a admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:48.720471 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7d663c-8c9f-409c-b5b6-8d76c9bd572a admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:48.721098 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7d663c-8c9f-409c-b5b6-8d76c9bd572a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:48.723017 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7d663c-8c9f-409c-b5b6-8d76c9bd572a admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:48.754054 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1269/2563] 10.222.0.102 () {60 vars in 1158 bytes} [Thu Sep 1 17:39:48 2022] POST /identity/v3/projects => generated 388 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:39:48.758137 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cda465-751f-4df9-9436-27f6f4fe957c admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:48.768173 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cda465-751f-4df9-9436-27f6f4fe957c admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:48.795774 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:39:48.829854 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e87881-c7ed-4555-8b5a-b41df4b99604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:48.830463 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e87881-c7ed-4555-8b5a-b41df4b99604 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:48.830743 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e87881-c7ed-4555-8b5a-b41df4b99604 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:48.831015 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e87881-c7ed-4555-8b5a-b41df4b99604 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:48.831644 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e87881-c7ed-4555-8b5a-b41df4b99604 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:48.833639 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e87881-c7ed-4555-8b5a-b41df4b99604 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:48.863481 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1295/2564] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:39:48 2022] POST /identity/v3/users => generated 295 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:39:48.871368 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7d663c-8c9f-409c-b5b6-8d76c9bd572a admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:48.883254 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7d663c-8c9f-409c-b5b6-8d76c9bd572a admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:48.887821 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d57f41-3300-42bb-a888-399a84b5f63c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:48.888409 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d57f41-3300-42bb-a888-399a84b5f63c admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:48.888682 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d57f41-3300-42bb-a888-399a84b5f63c admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:48.889150 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d57f41-3300-42bb-a888-399a84b5f63c admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:48.889606 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d57f41-3300-42bb-a888-399a84b5f63c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:48.891188 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d57f41-3300-42bb-a888-399a84b5f63c admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:48.901241 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1270/2565] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:39:48 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:39:48.905592 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e87881-c7ed-4555-8b5a-b41df4b99604 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:48.916172 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e87881-c7ed-4555-8b5a-b41df4b99604 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:48.920835 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63ef3a2-cb12-408d-850a-27ad1d13d2fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:48.921462 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63ef3a2-cb12-408d-850a-27ad1d13d2fe admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:48.921774 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63ef3a2-cb12-408d-850a-27ad1d13d2fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:48.922053 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63ef3a2-cb12-408d-850a-27ad1d13d2fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:48.922692 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63ef3a2-cb12-408d-850a-27ad1d13d2fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:48.924285 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63ef3a2-cb12-408d-850a-27ad1d13d2fe admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:48.935455 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1296/2566] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:39:48 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:39:48.939521 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d57f41-3300-42bb-a888-399a84b5f63c admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:48.949236 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d57f41-3300-42bb-a888-399a84b5f63c admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:48.954002 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a83db2b-a9e4-4207-a75f-7e5dc2ff9efc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:48.954741 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a83db2b-a9e4-4207-a75f-7e5dc2ff9efc admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:48.955019 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a83db2b-a9e4-4207-a75f-7e5dc2ff9efc admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:48.955287 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a83db2b-a9e4-4207-a75f-7e5dc2ff9efc admin admin] PATH_INFO: `/v3/projects/8f7cf084e40e4342ac76035a6e782c7e/users/b3702863093d447c94a1adf4aed4170b/roles/b75cb360925e4ba5a65f14965c103ffd` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:48.969264 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a83db2b-a9e4-4207-a75f-7e5dc2ff9efc admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f7cf084e40e4342ac76035a6e782c7e, user_id=b3702863093d447c94a1adf4aed4170b, role_id=b75cb360925e4ba5a65f14965c103ffd)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:48.971931 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a83db2b-a9e4-4207-a75f-7e5dc2ff9efc admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:48.984961 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1271/2567] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:39:48 2022] PUT /identity/v3/projects/8f7cf084e40e4342ac76035a6e782c7e/users/b3702863093d447c94a1adf4aed4170b/roles/b75cb360925e4ba5a65f14965c103ffd => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:39:48.996737 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe543da-63fe-4698-a35c-f6e5e17609f7 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:48.997025 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe543da-63fe-4698-a35c-f6e5e17609f7 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:48.997300 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe543da-63fe-4698-a35c-f6e5e17609f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:49.026564 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-4fe543da-63fe-4698-a35c-f6e5e17609f7 None None] MFA Rules not processed for user `b3702863093d447c94a1adf4aed4170b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:39:49.027792 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-4fe543da-63fe-4698-a35c-f6e5e17609f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:39:49.091812 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1297/2568] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:39:48 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:39:49.096380 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a83db2b-a9e4-4207-a75f-7e5dc2ff9efc admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:49.157045 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a83db2b-a9e4-4207-a75f-7e5dc2ff9efc admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:49.162119 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494c671e-dcbf-44ac-9e80-8a52ab804d4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:49.162711 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494c671e-dcbf-44ac-9e80-8a52ab804d4f admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:49.162993 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494c671e-dcbf-44ac-9e80-8a52ab804d4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:49.163268 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494c671e-dcbf-44ac-9e80-8a52ab804d4f admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:49.163902 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494c671e-dcbf-44ac-9e80-8a52ab804d4f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:49.165841 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494c671e-dcbf-44ac-9e80-8a52ab804d4f admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:49.192876 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1272/2569] 10.222.0.102 () {60 vars in 1158 bytes} [Thu Sep 1 17:39:49 2022] POST /identity/v3/projects => generated 388 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:39:49.199983 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe543da-63fe-4698-a35c-f6e5e17609f7 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:49.211742 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe543da-63fe-4698-a35c-f6e5e17609f7 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:49.271548 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c082c4-c81e-4995-9f1d-7e04a61ab4f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:49.272154 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c082c4-c81e-4995-9f1d-7e04a61ab4f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:49.272433 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c082c4-c81e-4995-9f1d-7e04a61ab4f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:49.272705 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c082c4-c81e-4995-9f1d-7e04a61ab4f6 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:49.273353 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c082c4-c81e-4995-9f1d-7e04a61ab4f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:49.275328 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c082c4-c81e-4995-9f1d-7e04a61ab4f6 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:49.296396 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1298/2570] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:39:49 2022] POST /identity/v3/users => generated 301 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:39:49.300528 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494c671e-dcbf-44ac-9e80-8a52ab804d4f admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:49.310730 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494c671e-dcbf-44ac-9e80-8a52ab804d4f admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:49.315330 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36af27a1-c449-40a9-98cc-8adbe0be92f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:49.315920 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36af27a1-c449-40a9-98cc-8adbe0be92f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:49.316206 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36af27a1-c449-40a9-98cc-8adbe0be92f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:49.316480 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36af27a1-c449-40a9-98cc-8adbe0be92f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:49.317117 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36af27a1-c449-40a9-98cc-8adbe0be92f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:49.318739 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36af27a1-c449-40a9-98cc-8adbe0be92f8 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:49.329683 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1273/2571] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:39:49 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 17:39:49.334204 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c082c4-c81e-4995-9f1d-7e04a61ab4f6 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:49.343995 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c082c4-c81e-4995-9f1d-7e04a61ab4f6 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:49.349198 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751128b2-28ba-464a-998a-a853ecbed6bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:49.349873 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751128b2-28ba-464a-998a-a853ecbed6bc admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:49.350155 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751128b2-28ba-464a-998a-a853ecbed6bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:49.350435 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751128b2-28ba-464a-998a-a853ecbed6bc admin admin] PATH_INFO: `/v3/projects/b7253cfb7b76408ebcd9f4c1f53cbba8/users/34e465f1dcdd4f0585824792903791f7/roles/36577a6f46d642d4b81805ede9872490` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:49.365881 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751128b2-28ba-464a-998a-a853ecbed6bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7253cfb7b76408ebcd9f4c1f53cbba8, user_id=34e465f1dcdd4f0585824792903791f7, role_id=36577a6f46d642d4b81805ede9872490)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:49.368519 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751128b2-28ba-464a-998a-a853ecbed6bc admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:49.380857 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1299/2572] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:39:49 2022] PUT /identity/v3/projects/b7253cfb7b76408ebcd9f4c1f53cbba8/users/34e465f1dcdd4f0585824792903791f7/roles/36577a6f46d642d4b81805ede9872490 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:39:49.384723 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36af27a1-c449-40a9-98cc-8adbe0be92f8 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:49.394516 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36af27a1-c449-40a9-98cc-8adbe0be92f8 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:49.414804 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:39:49.453357 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a186be0-6e09-4a60-a265-b3b7e40ecd45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:49.453990 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a186be0-6e09-4a60-a265-b3b7e40ecd45 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:49.454220 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a186be0-6e09-4a60-a265-b3b7e40ecd45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:49.454495 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a186be0-6e09-4a60-a265-b3b7e40ecd45 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:49.455122 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a186be0-6e09-4a60-a265-b3b7e40ecd45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:49.456704 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a186be0-6e09-4a60-a265-b3b7e40ecd45 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:49.466432 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1274/2573] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:39:49 2022] GET /identity/v3/roles => generated 1427 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:39:49.471749 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751128b2-28ba-464a-998a-a853ecbed6bc admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:49.477757 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:39:49.483717 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751128b2-28ba-464a-998a-a853ecbed6bc admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:49.488701 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c9dc22-dbf9-4f9f-804c-18ae2d61122f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:49.490002 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c9dc22-dbf9-4f9f-804c-18ae2d61122f admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:49.490278 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c9dc22-dbf9-4f9f-804c-18ae2d61122f admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:49.490546 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c9dc22-dbf9-4f9f-804c-18ae2d61122f admin admin] PATH_INFO: `/v3/projects/b7253cfb7b76408ebcd9f4c1f53cbba8/users/34e465f1dcdd4f0585824792903791f7/roles/36577a6f46d642d4b81805ede9872490` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:49.492421 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c9dc22-dbf9-4f9f-804c-18ae2d61122f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7253cfb7b76408ebcd9f4c1f53cbba8, user_id=34e465f1dcdd4f0585824792903791f7, role_id=36577a6f46d642d4b81805ede9872490)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:49.495064 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c9dc22-dbf9-4f9f-804c-18ae2d61122f admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:49.505021 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1300/2574] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:39:49 2022] PUT /identity/v3/projects/b7253cfb7b76408ebcd9f4c1f53cbba8/users/34e465f1dcdd4f0585824792903791f7/roles/36577a6f46d642d4b81805ede9872490 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:39:49.514994 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a5fc6d-d464-483b-9d29-785e829f71d2 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:49.515283 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a5fc6d-d464-483b-9d29-785e829f71d2 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:49.515561 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a5fc6d-d464-483b-9d29-785e829f71d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:49.543317 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-43a5fc6d-d464-483b-9d29-785e829f71d2 None None] MFA Rules not processed for user `34e465f1dcdd4f0585824792903791f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:39:49.544534 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-43a5fc6d-d464-483b-9d29-785e829f71d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:39:49.612149 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1275/2575] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:39:49 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:39:49.623700 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c9dc22-dbf9-4f9f-804c-18ae2d61122f admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:49.625515 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-e2c9dc22-dbf9-4f9f-804c-18ae2d61122f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:39:49.682485 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c9dc22-dbf9-4f9f-804c-18ae2d61122f admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:49.686138 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b866ec-10df-4cad-b4bd-0153f1494247 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '523ed589872a45a09b6a7675a7fb0b75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:49.686731 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b866ec-10df-4cad-b4bd-0153f1494247 service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:49.687010 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b866ec-10df-4cad-b4bd-0153f1494247 service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:49.687282 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b866ec-10df-4cad-b4bd-0153f1494247 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:49.696120 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b866ec-10df-4cad-b4bd-0153f1494247 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:49.697569 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b866ec-10df-4cad-b4bd-0153f1494247 service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:49.707051 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1301/2576] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:39:49 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:39:49.840659 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a5fc6d-d464-483b-9d29-785e829f71d2 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:49.914317 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a5fc6d-d464-483b-9d29-785e829f71d2 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:49.919071 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e5a8f2-839c-48e3-9a1d-cd3b392f541c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:49.919694 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e5a8f2-839c-48e3-9a1d-cd3b392f541c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:49.919974 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e5a8f2-839c-48e3-9a1d-cd3b392f541c admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:49.920249 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e5a8f2-839c-48e3-9a1d-cd3b392f541c admin admin] PATH_INFO: `/v3/users/b3702863093d447c94a1adf4aed4170b` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:49.921428 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e5a8f2-839c-48e3-9a1d-cd3b392f541c admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3702863093d447c94a1adf4aed4170b)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:49.923341 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e5a8f2-839c-48e3-9a1d-cd3b392f541c admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:49.971586 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-94e5a8f2-839c-48e3-9a1d-cd3b392f541c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3702863093d447c94a1adf4aed4170b'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:49.972318 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-94e5a8f2-839c-48e3-9a1d-cd3b392f541c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3702863093d447c94a1adf4aed4170b'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:49.974312 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:39:49.991140 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-94e5a8f2-839c-48e3-9a1d-cd3b392f541c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3702863093d447c94a1adf4aed4170b'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:49.998793 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-94e5a8f2-839c-48e3-9a1d-cd3b392f541c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3702863093d447c94a1adf4aed4170b'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:49.999426 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-94e5a8f2-839c-48e3-9a1d-cd3b392f541c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3702863093d447c94a1adf4aed4170b'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:50.011506 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1276/2577] 10.222.0.102 () {58 vars in 1229 bytes} [Thu Sep 1 17:39:49 2022] DELETE /identity/v3/users/b3702863093d447c94a1adf4aed4170b => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:39:50.015941 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa64821-71a1-412e-ba85-2585f8116c75 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:50.016234 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa64821-71a1-412e-ba85-2585f8116c75 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:50.016511 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa64821-71a1-412e-ba85-2585f8116c75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:50.036706 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-faa64821-71a1-412e-ba85-2585f8116c75 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:39:50.037926 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-faa64821-71a1-412e-ba85-2585f8116c75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:39:50.094623 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1302/2578] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:39:50 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:39:50.104870 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e5a8f2-839c-48e3-9a1d-cd3b392f541c admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:50.106295 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-94e5a8f2-839c-48e3-9a1d-cd3b392f541c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:39:50.162990 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e5a8f2-839c-48e3-9a1d-cd3b392f541c admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:50.166592 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc60d1b-66b4-4204-8a33-aa9692c41eb4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:50.167185 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc60d1b-66b4-4204-8a33-aa9692c41eb4 service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:50.167467 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc60d1b-66b4-4204-8a33-aa9692c41eb4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:50.167735 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc60d1b-66b4-4204-8a33-aa9692c41eb4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:50.176487 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc60d1b-66b4-4204-8a33-aa9692c41eb4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:50.177931 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc60d1b-66b4-4204-8a33-aa9692c41eb4 service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:50.187000 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1277/2579] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:39:50 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:39:50.460662 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa64821-71a1-412e-ba85-2585f8116c75 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:50.462352 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-faa64821-71a1-412e-ba85-2585f8116c75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:39:50.474955 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa64821-71a1-412e-ba85-2585f8116c75 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:50.479572 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31ac4b9-3172-45e8-a940-5df72f47dbe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:50.480206 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31ac4b9-3172-45e8-a940-5df72f47dbe9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:50.480490 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31ac4b9-3172-45e8-a940-5df72f47dbe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:50.480765 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31ac4b9-3172-45e8-a940-5df72f47dbe9 admin admin] PATH_INFO: `/v3/projects/8f7cf084e40e4342ac76035a6e782c7e` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:50.481919 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31ac4b9-3172-45e8-a940-5df72f47dbe9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f7cf084e40e4342ac76035a6e782c7e)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:50.483862 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31ac4b9-3172-45e8-a940-5df72f47dbe9 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:50.530696 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-d31ac4b9-3172-45e8-a940-5df72f47dbe9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f7cf084e40e4342ac76035a6e782c7e', 'request_id': 'req-d31ac4b9-3172-45e8-a940-5df72f47dbe9'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:50.538732 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-d31ac4b9-3172-45e8-a940-5df72f47dbe9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f7cf084e40e4342ac76035a6e782c7e', 'request_id': 'req-d31ac4b9-3172-45e8-a940-5df72f47dbe9'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:50.545181 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-d31ac4b9-3172-45e8-a940-5df72f47dbe9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f7cf084e40e4342ac76035a6e782c7e', 'request_id': 'req-d31ac4b9-3172-45e8-a940-5df72f47dbe9'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:50.556033 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-d31ac4b9-3172-45e8-a940-5df72f47dbe9 admin admin] The token cache is being invalidate because project 8f7cf084e40e4342ac76035a6e782c7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:39:50.556320 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-d31ac4b9-3172-45e8-a940-5df72f47dbe9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:50.556941 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-d31ac4b9-3172-45e8-a940-5df72f47dbe9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:50.558489 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1303/2580] 10.222.0.102 () {58 vars in 1238 bytes} [Thu Sep 1 17:39:50 2022] DELETE /identity/v3/projects/8f7cf084e40e4342ac76035a6e782c7e => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:39:50.562466 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc60d1b-66b4-4204-8a33-aa9692c41eb4 service neutron] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:50.572692 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc60d1b-66b4-4204-8a33-aa9692c41eb4 service neutron] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:50.629858 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85a7cd5-d2d3-4e15-800d-7036816974dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:50.630474 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85a7cd5-d2d3-4e15-800d-7036816974dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:50.630753 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85a7cd5-d2d3-4e15-800d-7036816974dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:50.631025 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85a7cd5-d2d3-4e15-800d-7036816974dc admin admin] PATH_INFO: `/v3/users/34e465f1dcdd4f0585824792903791f7` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:50.632131 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85a7cd5-d2d3-4e15-800d-7036816974dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=34e465f1dcdd4f0585824792903791f7)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:50.634076 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85a7cd5-d2d3-4e15-800d-7036816974dc admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:50.678702 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-d85a7cd5-d2d3-4e15-800d-7036816974dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34e465f1dcdd4f0585824792903791f7'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:50.679336 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-d85a7cd5-d2d3-4e15-800d-7036816974dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34e465f1dcdd4f0585824792903791f7'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:50.691268 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-d85a7cd5-d2d3-4e15-800d-7036816974dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34e465f1dcdd4f0585824792903791f7'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:50.694871 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:39:50.699192 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-d85a7cd5-d2d3-4e15-800d-7036816974dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34e465f1dcdd4f0585824792903791f7'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:50.700261 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-d85a7cd5-d2d3-4e15-800d-7036816974dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34e465f1dcdd4f0585824792903791f7'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:50.715049 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1278/2581] 10.222.0.102 () {58 vars in 1229 bytes} [Thu Sep 1 17:39:50 2022] DELETE /identity/v3/users/34e465f1dcdd4f0585824792903791f7 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:39:50.986928 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31ac4b9-3172-45e8-a940-5df72f47dbe9 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:50.988409 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-d31ac4b9-3172-45e8-a940-5df72f47dbe9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:39:51.005344 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31ac4b9-3172-45e8-a940-5df72f47dbe9 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:51.019505 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:39:51.072037 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaae291-d964-4076-b355-b4fe0392258b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:51.072609 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaae291-d964-4076-b355-b4fe0392258b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:51.072887 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaae291-d964-4076-b355-b4fe0392258b admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:51.073158 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaae291-d964-4076-b355-b4fe0392258b admin admin] PATH_INFO: `/v3/projects/b7253cfb7b76408ebcd9f4c1f53cbba8` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:51.074328 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaae291-d964-4076-b355-b4fe0392258b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7253cfb7b76408ebcd9f4c1f53cbba8)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:51.076599 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaae291-d964-4076-b355-b4fe0392258b admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:51.126662 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-8eaae291-d964-4076-b355-b4fe0392258b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7253cfb7b76408ebcd9f4c1f53cbba8', 'request_id': 'req-8eaae291-d964-4076-b355-b4fe0392258b'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:51.134444 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-8eaae291-d964-4076-b355-b4fe0392258b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7253cfb7b76408ebcd9f4c1f53cbba8', 'request_id': 'req-8eaae291-d964-4076-b355-b4fe0392258b'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:51.140909 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-8eaae291-d964-4076-b355-b4fe0392258b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7253cfb7b76408ebcd9f4c1f53cbba8', 'request_id': 'req-8eaae291-d964-4076-b355-b4fe0392258b'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:51.152118 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-8eaae291-d964-4076-b355-b4fe0392258b admin admin] The token cache is being invalidate because project b7253cfb7b76408ebcd9f4c1f53cbba8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:39:51.152423 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-8eaae291-d964-4076-b355-b4fe0392258b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:51.153122 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-8eaae291-d964-4076-b355-b4fe0392258b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:51.154836 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1304/2582] 10.222.0.102 () {58 vars in 1238 bytes} [Thu Sep 1 17:39:50 2022] DELETE /identity/v3/projects/b7253cfb7b76408ebcd9f4c1f53cbba8 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:39:51.159614 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd615d4-232a-4733-91c4-78ca7b68bc53 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:51.159970 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd615d4-232a-4733-91c4-78ca7b68bc53 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:51.160422 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd615d4-232a-4733-91c4-78ca7b68bc53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:51.184969 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-ddd615d4-232a-4733-91c4-78ca7b68bc53 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:39:51.186227 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-ddd615d4-232a-4733-91c4-78ca7b68bc53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:39:51.272658 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1279/2583] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:39:51 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:39:51.280615 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a306d61-bdd2-45f0-89a7-04a8319bfc43 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:51.280937 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a306d61-bdd2-45f0-89a7-04a8319bfc43 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:51.281219 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a306d61-bdd2-45f0-89a7-04a8319bfc43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:51.300702 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-3a306d61-bdd2-45f0-89a7-04a8319bfc43 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:39:51.301946 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-3a306d61-bdd2-45f0-89a7-04a8319bfc43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:39:51.309866 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1305/2584] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:39:51 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:39:51.313858 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd615d4-232a-4733-91c4-78ca7b68bc53 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:51.324394 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd615d4-232a-4733-91c4-78ca7b68bc53 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:51.329123 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f166a5c-f5a2-416d-b67e-e2fa1f5e577c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:51.329734 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f166a5c-f5a2-416d-b67e-e2fa1f5e577c admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:51.330010 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f166a5c-f5a2-416d-b67e-e2fa1f5e577c admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:51.330278 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f166a5c-f5a2-416d-b67e-e2fa1f5e577c admin admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:51.330916 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f166a5c-f5a2-416d-b67e-e2fa1f5e577c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:39:51.331215 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f166a5c-f5a2-416d-b67e-e2fa1f5e577c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:51.333474 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f166a5c-f5a2-416d-b67e-e2fa1f5e577c admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:51.345215 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1280/2585] 10.222.0.102 () {58 vars in 1158 bytes} [Thu Sep 1 17:39:51 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:39:51.349441 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a306d61-bdd2-45f0-89a7-04a8319bfc43 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:51.358537 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a306d61-bdd2-45f0-89a7-04a8319bfc43 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:51.363184 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0093d44a-9802-4a0b-812b-f84522721510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:51.363778 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0093d44a-9802-4a0b-812b-f84522721510 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:51.364546 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0093d44a-9802-4a0b-812b-f84522721510 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:51.364813 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0093d44a-9802-4a0b-812b-f84522721510 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:51.365487 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0093d44a-9802-4a0b-812b-f84522721510 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:51.367375 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0093d44a-9802-4a0b-812b-f84522721510 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:51.394257 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1306/2586] 10.222.0.102 () {60 vars in 1158 bytes} [Thu Sep 1 17:39:51 2022] POST /identity/v3/projects => generated 372 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:39:51.398588 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f166a5c-f5a2-416d-b67e-e2fa1f5e577c admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:51.408884 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f166a5c-f5a2-416d-b67e-e2fa1f5e577c admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:51.465739 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647fb802-80d5-4d15-bdea-8a975011103d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:51.466343 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647fb802-80d5-4d15-bdea-8a975011103d admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:51.466620 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647fb802-80d5-4d15-bdea-8a975011103d admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:51.466892 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647fb802-80d5-4d15-bdea-8a975011103d admin admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:51.467529 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647fb802-80d5-4d15-bdea-8a975011103d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:51.469466 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647fb802-80d5-4d15-bdea-8a975011103d admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:51.492461 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1281/2587] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:39:51 2022] POST /identity/v3/users => generated 287 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:39:51.496630 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0093d44a-9802-4a0b-812b-f84522721510 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:51.508081 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0093d44a-9802-4a0b-812b-f84522721510 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:51.512735 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b08f864-0cba-45d6-96e9-16c1d707d1dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:51.513355 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b08f864-0cba-45d6-96e9-16c1d707d1dc admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:51.513641 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b08f864-0cba-45d6-96e9-16c1d707d1dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:51.513914 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b08f864-0cba-45d6-96e9-16c1d707d1dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:51.514552 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b08f864-0cba-45d6-96e9-16c1d707d1dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:51.516185 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b08f864-0cba-45d6-96e9-16c1d707d1dc admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:51.529805 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1307/2588] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:39:51 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:39:51.531116 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647fb802-80d5-4d15-bdea-8a975011103d admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:51.541193 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647fb802-80d5-4d15-bdea-8a975011103d admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:51.551159 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b642f25b-46f5-4ac8-993d-7e9aefdb290c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:51.552704 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b642f25b-46f5-4ac8-993d-7e9aefdb290c admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:51.553175 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b642f25b-46f5-4ac8-993d-7e9aefdb290c admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:51.553677 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b642f25b-46f5-4ac8-993d-7e9aefdb290c admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:51.554816 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b642f25b-46f5-4ac8-993d-7e9aefdb290c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:51.557759 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b642f25b-46f5-4ac8-993d-7e9aefdb290c admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:51.570792 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1282/2589] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:39:51 2022] GET /identity/v3/roles => generated 1427 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:39:51.578577 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b08f864-0cba-45d6-96e9-16c1d707d1dc admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:51.587561 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b08f864-0cba-45d6-96e9-16c1d707d1dc admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:51.593241 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83ad62e-653e-44ac-8371-8e3c8ce18e27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:51.594016 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83ad62e-653e-44ac-8371-8e3c8ce18e27 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:51.594368 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83ad62e-653e-44ac-8371-8e3c8ce18e27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:51.594733 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83ad62e-653e-44ac-8371-8e3c8ce18e27 admin admin] PATH_INFO: `/v3/projects/a2e8bfe6cd764cacb3671b56f143194a/users/08f476a0c46d4343955fd35a7b29c650/roles/b75cb360925e4ba5a65f14965c103ffd` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:51.609424 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83ad62e-653e-44ac-8371-8e3c8ce18e27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2e8bfe6cd764cacb3671b56f143194a, user_id=08f476a0c46d4343955fd35a7b29c650, role_id=b75cb360925e4ba5a65f14965c103ffd)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:51.612081 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83ad62e-653e-44ac-8371-8e3c8ce18e27 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:51.625550 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1308/2590] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:39:51 2022] PUT /identity/v3/projects/a2e8bfe6cd764cacb3671b56f143194a/users/08f476a0c46d4343955fd35a7b29c650/roles/b75cb360925e4ba5a65f14965c103ffd => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:39:51.693727 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc82507-9e79-4755-a86c-21727ac196c2 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:51.694040 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc82507-9e79-4755-a86c-21727ac196c2 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:51.694313 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc82507-9e79-4755-a86c-21727ac196c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:51.722726 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-6bc82507-9e79-4755-a86c-21727ac196c2 None None] MFA Rules not processed for user `08f476a0c46d4343955fd35a7b29c650`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:39:51.724462 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-6bc82507-9e79-4755-a86c-21727ac196c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:39:51.773380 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:39:51.803726 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1283/2591] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:39:51 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:39:51.822594 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83ad62e-653e-44ac-8371-8e3c8ce18e27 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:51.824316 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-f83ad62e-653e-44ac-8371-8e3c8ce18e27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:39:51.878623 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:39:51.892829 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83ad62e-653e-44ac-8371-8e3c8ce18e27 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:51.898128 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3abce4-99be-434f-a87e-8c6ff30ba3ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '523ed589872a45a09b6a7675a7fb0b75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:51.898766 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3abce4-99be-434f-a87e-8c6ff30ba3ce service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:51.899048 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3abce4-99be-434f-a87e-8c6ff30ba3ce service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:51.899576 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3abce4-99be-434f-a87e-8c6ff30ba3ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:51.910373 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3abce4-99be-434f-a87e-8c6ff30ba3ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:51.911944 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3abce4-99be-434f-a87e-8c6ff30ba3ce service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:51.921595 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1309/2592] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:39:51 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:39:51.942979 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc82507-9e79-4755-a86c-21727ac196c2 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:52.009456 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc82507-9e79-4755-a86c-21727ac196c2 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:52.015100 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae79950d-9290-4b2f-8bc3-0229a1ec93c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:52.015732 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae79950d-9290-4b2f-8bc3-0229a1ec93c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:52.016008 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae79950d-9290-4b2f-8bc3-0229a1ec93c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:52.016278 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae79950d-9290-4b2f-8bc3-0229a1ec93c2 admin admin] PATH_INFO: `/v3/users/08f476a0c46d4343955fd35a7b29c650` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:52.017474 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae79950d-9290-4b2f-8bc3-0229a1ec93c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=08f476a0c46d4343955fd35a7b29c650)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:52.019827 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae79950d-9290-4b2f-8bc3-0229a1ec93c2 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:52.025868 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3abce4-99be-434f-a87e-8c6ff30ba3ce service cinder] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:52.027945 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-0e3abce4-99be-434f-a87e-8c6ff30ba3ce service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:39:52.068536 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-ae79950d-9290-4b2f-8bc3-0229a1ec93c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08f476a0c46d4343955fd35a7b29c650'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:52.069198 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-ae79950d-9290-4b2f-8bc3-0229a1ec93c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08f476a0c46d4343955fd35a7b29c650'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:52.082211 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-ae79950d-9290-4b2f-8bc3-0229a1ec93c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08f476a0c46d4343955fd35a7b29c650'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:52.090330 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-ae79950d-9290-4b2f-8bc3-0229a1ec93c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08f476a0c46d4343955fd35a7b29c650'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:52.091033 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-ae79950d-9290-4b2f-8bc3-0229a1ec93c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08f476a0c46d4343955fd35a7b29c650'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:52.103556 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3abce4-99be-434f-a87e-8c6ff30ba3ce service cinder] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:52.106866 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1284/2593] 10.222.0.102 () {58 vars in 1229 bytes} [Thu Sep 1 17:39:51 2022] DELETE /identity/v3/users/08f476a0c46d4343955fd35a7b29c650 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:39:52.108540 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5218afe-a330-42d8-b4a7-27750d891281 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:52.108849 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5218afe-a330-42d8-b4a7-27750d891281 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:52.109149 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5218afe-a330-42d8-b4a7-27750d891281 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:52.128951 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-c5218afe-a330-42d8-b4a7-27750d891281 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:39:52.131395 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-c5218afe-a330-42d8-b4a7-27750d891281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:39:52.188019 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e21664-6154-41f6-8217-644be1f06be0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '134ef6039cef468bae0e3f3be79b8073', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:52.188716 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e21664-6154-41f6-8217-644be1f06be0 service placement] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:52.188917 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e21664-6154-41f6-8217-644be1f06be0 service placement] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:52.189191 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e21664-6154-41f6-8217-644be1f06be0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:52.191459 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-76e21664-6154-41f6-8217-644be1f06be0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:39:52.205633 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e21664-6154-41f6-8217-644be1f06be0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:52.207249 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e21664-6154-41f6-8217-644be1f06be0 service placement] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:52.216482 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1285/2594] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:39:52 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:39:52.219503 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1310/2595] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:39:52 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 17:39:52.225377 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5218afe-a330-42d8-b4a7-27750d891281 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:52.226973 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-c5218afe-a330-42d8-b4a7-27750d891281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:39:52.232836 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:39:52.310945 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5218afe-a330-42d8-b4a7-27750d891281 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:52.315130 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd90752-acf2-4c0f-848f-84ce3b4fcc2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:52.315745 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd90752-acf2-4c0f-848f-84ce3b4fcc2a service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:52.316034 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd90752-acf2-4c0f-848f-84ce3b4fcc2a service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:52.316296 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd90752-acf2-4c0f-848f-84ce3b4fcc2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:52.326514 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd90752-acf2-4c0f-848f-84ce3b4fcc2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:52.327938 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd90752-acf2-4c0f-848f-84ce3b4fcc2a service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:52.337216 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1286/2596] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:39:52 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 17:39:52.608780 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e21664-6154-41f6-8217-644be1f06be0 service placement] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:52.610363 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-76e21664-6154-41f6-8217-644be1f06be0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:39:52.630770 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e21664-6154-41f6-8217-644be1f06be0 service placement] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:52.636012 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ce6a98-cd4b-4653-b615-62417034ccfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:52.636702 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ce6a98-cd4b-4653-b615-62417034ccfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:52.637011 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ce6a98-cd4b-4653-b615-62417034ccfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:52.637286 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ce6a98-cd4b-4653-b615-62417034ccfe admin admin] PATH_INFO: `/v3/projects/a2e8bfe6cd764cacb3671b56f143194a` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:52.638438 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ce6a98-cd4b-4653-b615-62417034ccfe admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2e8bfe6cd764cacb3671b56f143194a)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:52.640567 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ce6a98-cd4b-4653-b615-62417034ccfe admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:52.687146 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-06ce6a98-cd4b-4653-b615-62417034ccfe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2e8bfe6cd764cacb3671b56f143194a', 'request_id': 'req-06ce6a98-cd4b-4653-b615-62417034ccfe'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:52.694254 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-06ce6a98-cd4b-4653-b615-62417034ccfe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2e8bfe6cd764cacb3671b56f143194a', 'request_id': 'req-06ce6a98-cd4b-4653-b615-62417034ccfe'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:52.701098 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-06ce6a98-cd4b-4653-b615-62417034ccfe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2e8bfe6cd764cacb3671b56f143194a', 'request_id': 'req-06ce6a98-cd4b-4653-b615-62417034ccfe'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:52.708031 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:39:52.711917 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-06ce6a98-cd4b-4653-b615-62417034ccfe admin admin] The token cache is being invalidate because project a2e8bfe6cd764cacb3671b56f143194a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:39:52.712542 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-06ce6a98-cd4b-4653-b615-62417034ccfe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:52.713298 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-06ce6a98-cd4b-4653-b615-62417034ccfe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:52.714927 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1311/2597] 10.222.0.102 () {58 vars in 1238 bytes} [Thu Sep 1 17:39:52 2022] DELETE /identity/v3/projects/a2e8bfe6cd764cacb3671b56f143194a => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:39:52.720894 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144bad40-63e5-4d55-9d8f-747bcfe4a068 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:52.721400 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144bad40-63e5-4d55-9d8f-747bcfe4a068 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:52.721531 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144bad40-63e5-4d55-9d8f-747bcfe4a068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:52.741557 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-144bad40-63e5-4d55-9d8f-747bcfe4a068 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:39:52.742740 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-144bad40-63e5-4d55-9d8f-747bcfe4a068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:39:52.800549 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1287/2598] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:39:52 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:39:52.808592 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0ab9eb-ff95-45bd-9449-0e3df7124fd6 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:52.808954 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0ab9eb-ff95-45bd-9449-0e3df7124fd6 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:52.809279 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0ab9eb-ff95-45bd-9449-0e3df7124fd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:52.829563 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-5f0ab9eb-ff95-45bd-9449-0e3df7124fd6 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:39:52.830763 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-5f0ab9eb-ff95-45bd-9449-0e3df7124fd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:39:52.839147 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1312/2599] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:39:52 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:39:52.843442 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144bad40-63e5-4d55-9d8f-747bcfe4a068 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:52.856126 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144bad40-63e5-4d55-9d8f-747bcfe4a068 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:52.862403 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef042e7-fab7-4309-a3d5-b4891ea26f0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:52.863141 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef042e7-fab7-4309-a3d5-b4891ea26f0e admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:52.863473 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef042e7-fab7-4309-a3d5-b4891ea26f0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:52.863756 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef042e7-fab7-4309-a3d5-b4891ea26f0e admin admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:52.864488 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef042e7-fab7-4309-a3d5-b4891ea26f0e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:39:52.864800 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef042e7-fab7-4309-a3d5-b4891ea26f0e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:52.866439 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef042e7-fab7-4309-a3d5-b4891ea26f0e admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:52.879212 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1288/2600] 10.222.0.102 () {58 vars in 1158 bytes} [Thu Sep 1 17:39:52 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:39:52.883835 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0ab9eb-ff95-45bd-9449-0e3df7124fd6 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:52.893351 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0ab9eb-ff95-45bd-9449-0e3df7124fd6 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:52.898638 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa1c061-ad4b-4b25-9600-a889a1d1e652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:52.899322 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa1c061-ad4b-4b25-9600-a889a1d1e652 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:52.899640 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa1c061-ad4b-4b25-9600-a889a1d1e652 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:52.899946 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa1c061-ad4b-4b25-9600-a889a1d1e652 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:52.900584 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa1c061-ad4b-4b25-9600-a889a1d1e652 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:52.902529 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa1c061-ad4b-4b25-9600-a889a1d1e652 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:52.935733 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1313/2601] 10.222.0.102 () {60 vars in 1158 bytes} [Thu Sep 1 17:39:52 2022] POST /identity/v3/projects => generated 376 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:39:52.936896 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef042e7-fab7-4309-a3d5-b4891ea26f0e admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:52.947131 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef042e7-fab7-4309-a3d5-b4891ea26f0e admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:53.009000 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d23857e-730d-40cc-befe-c66c42ffef6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:53.009634 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d23857e-730d-40cc-befe-c66c42ffef6a admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:53.009912 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d23857e-730d-40cc-befe-c66c42ffef6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:53.010181 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d23857e-730d-40cc-befe-c66c42ffef6a admin admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:53.010798 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d23857e-730d-40cc-befe-c66c42ffef6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:53.012706 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d23857e-730d-40cc-befe-c66c42ffef6a admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:53.034578 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1289/2602] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:39:52 2022] POST /identity/v3/users => generated 289 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:39:53.039348 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa1c061-ad4b-4b25-9600-a889a1d1e652 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:53.049804 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa1c061-ad4b-4b25-9600-a889a1d1e652 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:53.054418 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5b0476-66f5-4505-835e-c2997657bb40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:53.055004 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5b0476-66f5-4505-835e-c2997657bb40 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:53.055282 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5b0476-66f5-4505-835e-c2997657bb40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:53.055551 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5b0476-66f5-4505-835e-c2997657bb40 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:53.056177 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5b0476-66f5-4505-835e-c2997657bb40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:53.057890 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5b0476-66f5-4505-835e-c2997657bb40 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:53.067880 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1314/2603] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:39:53 2022] GET /identity/v3/roles => generated 1427 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:39:53.071887 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d23857e-730d-40cc-befe-c66c42ffef6a admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:53.082157 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d23857e-730d-40cc-befe-c66c42ffef6a admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:53.087246 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e763e46-9616-4553-b46f-d777bc0dc2bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:53.087827 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e763e46-9616-4553-b46f-d777bc0dc2bc admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:53.088184 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e763e46-9616-4553-b46f-d777bc0dc2bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:53.088463 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e763e46-9616-4553-b46f-d777bc0dc2bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:53.089121 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e763e46-9616-4553-b46f-d777bc0dc2bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:53.090721 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e763e46-9616-4553-b46f-d777bc0dc2bc admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:53.101781 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1290/2604] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:39:53 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:39:53.105896 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5b0476-66f5-4505-835e-c2997657bb40 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:53.115591 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5b0476-66f5-4505-835e-c2997657bb40 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:53.120092 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4180f6-2bd5-4508-93a2-ab8011d0e581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:53.120725 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4180f6-2bd5-4508-93a2-ab8011d0e581 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:53.121002 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4180f6-2bd5-4508-93a2-ab8011d0e581 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:53.121273 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4180f6-2bd5-4508-93a2-ab8011d0e581 admin admin] PATH_INFO: `/v3/projects/fd4ba94e5d9a4bf185bbbdf5e4c84ce8/users/44f3a9807c30494f93e52147967ddb3a/roles/b75cb360925e4ba5a65f14965c103ffd` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:53.135204 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4180f6-2bd5-4508-93a2-ab8011d0e581 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd4ba94e5d9a4bf185bbbdf5e4c84ce8, user_id=44f3a9807c30494f93e52147967ddb3a, role_id=b75cb360925e4ba5a65f14965c103ffd)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:53.137878 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4180f6-2bd5-4508-93a2-ab8011d0e581 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:53.151098 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1315/2605] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:39:53 2022] PUT /identity/v3/projects/fd4ba94e5d9a4bf185bbbdf5e4c84ce8/users/44f3a9807c30494f93e52147967ddb3a/roles/b75cb360925e4ba5a65f14965c103ffd => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:39:53.161186 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408c0010-d555-43e6-8c2f-18ecd348718f None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:53.161514 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408c0010-d555-43e6-8c2f-18ecd348718f None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:53.161793 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408c0010-d555-43e6-8c2f-18ecd348718f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:53.182231 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:39:53.190573 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-408c0010-d555-43e6-8c2f-18ecd348718f None None] MFA Rules not processed for user `44f3a9807c30494f93e52147967ddb3a`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:39:53.192011 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-408c0010-d555-43e6-8c2f-18ecd348718f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:39:53.255381 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1291/2606] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:39:53 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:39:53.265932 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4180f6-2bd5-4508-93a2-ab8011d0e581 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:53.267755 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-fa4180f6-2bd5-4508-93a2-ab8011d0e581 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:39:53.326134 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4180f6-2bd5-4508-93a2-ab8011d0e581 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:53.329960 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63907f88-e221-4a20-964b-61fe2579cfac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '523ed589872a45a09b6a7675a7fb0b75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:53.330565 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63907f88-e221-4a20-964b-61fe2579cfac service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:53.330848 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63907f88-e221-4a20-964b-61fe2579cfac service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:53.331162 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63907f88-e221-4a20-964b-61fe2579cfac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:53.340341 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63907f88-e221-4a20-964b-61fe2579cfac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:53.341815 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63907f88-e221-4a20-964b-61fe2579cfac service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:53.352420 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1316/2607] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:39:53 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:39:53.408751 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfede45-5ad5-4a95-be8d-4fb811a5d01b None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:53.409046 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfede45-5ad5-4a95-be8d-4fb811a5d01b None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:53.409352 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfede45-5ad5-4a95-be8d-4fb811a5d01b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:53.440036 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-1bfede45-5ad5-4a95-be8d-4fb811a5d01b None None] MFA Rules not processed for user `575a76b99ea945efb1024fbb3dc81aaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:39:53.441268 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-1bfede45-5ad5-4a95-be8d-4fb811a5d01b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:39:53.517062 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1292/2608] 10.222.0.102 () {58 vars in 972 bytes} [Thu Sep 1 17:39:53 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:39:53.570484 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63907f88-e221-4a20-964b-61fe2579cfac service cinder] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:53.571960 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-63907f88-e221-4a20-964b-61fe2579cfac service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:39:53.665328 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63907f88-e221-4a20-964b-61fe2579cfac service cinder] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:53.665328 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76962065-d5f6-49a8-965e-34cf818e71f7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd55edf6cd1804006aa8269b4c6f80aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:53.665328 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76962065-d5f6-49a8-965e-34cf818e71f7 service swift] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:53.665328 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76962065-d5f6-49a8-965e-34cf818e71f7 service swift] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:53.666157 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76962065-d5f6-49a8-965e-34cf818e71f7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:53.669966 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76962065-d5f6-49a8-965e-34cf818e71f7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:53.671623 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76962065-d5f6-49a8-965e-34cf818e71f7 service swift] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:53.680270 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1317/2609] 10.222.0.102 () {60 vars in 1389 bytes} [Thu Sep 1 17:39:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 17:39:54.733464 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e9b2af-0ad0-41f2-b894-3d0f6df9875c None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:54.734051 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e9b2af-0ad0-41f2-b894-3d0f6df9875c None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:54.734166 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e9b2af-0ad0-41f2-b894-3d0f6df9875c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:54.754437 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-b4e9b2af-0ad0-41f2-b894-3d0f6df9875c None None] MFA Rules not processed for user `44f3a9807c30494f93e52147967ddb3a`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:39:54.755788 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-b4e9b2af-0ad0-41f2-b894-3d0f6df9875c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:39:54.763174 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1293/2610] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:39:54 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:39:57.530926 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76962065-d5f6-49a8-965e-34cf818e71f7 service swift] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:57.537489 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-76962065-d5f6-49a8-965e-34cf818e71f7 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:39:57.614647 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:39:57.688118 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76962065-d5f6-49a8-965e-34cf818e71f7 service swift] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:57.693768 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998e12d1-2381-4cd3-9a3f-0d2f01794dbc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '134ef6039cef468bae0e3f3be79b8073', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:57.694712 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998e12d1-2381-4cd3-9a3f-0d2f01794dbc service placement] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:57.695007 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998e12d1-2381-4cd3-9a3f-0d2f01794dbc service placement] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:57.695279 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998e12d1-2381-4cd3-9a3f-0d2f01794dbc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:57.697255 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-998e12d1-2381-4cd3-9a3f-0d2f01794dbc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:39:57.727154 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998e12d1-2381-4cd3-9a3f-0d2f01794dbc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:57.728788 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998e12d1-2381-4cd3-9a3f-0d2f01794dbc service placement] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:57.743425 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1318/2611] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:39:57 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:40:02.122455 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e9b2af-0ad0-41f2-b894-3d0f6df9875c None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:40:02.124042 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-b4e9b2af-0ad0-41f2-b894-3d0f6df9875c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:40:02.188810 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e9b2af-0ad0-41f2-b894-3d0f6df9875c None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:40:02.193356 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3ed4ed-6dd1-42f7-b882-e596a253dfe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:40:02.194025 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3ed4ed-6dd1-42f7-b882-e596a253dfe8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:40:02.194258 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3ed4ed-6dd1-42f7-b882-e596a253dfe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:40:02.194534 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3ed4ed-6dd1-42f7-b882-e596a253dfe8 admin admin] PATH_INFO: `/v3/users/333dc785d4a84d56bd33a6b305dec2f7` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:40:02.195647 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3ed4ed-6dd1-42f7-b882-e596a253dfe8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=333dc785d4a84d56bd33a6b305dec2f7)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:40:02.197584 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3ed4ed-6dd1-42f7-b882-e596a253dfe8 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:40:02.225976 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:40:02.244425 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-8b3ed4ed-6dd1-42f7-b882-e596a253dfe8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '333dc785d4a84d56bd33a6b305dec2f7'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:40:02.245022 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-8b3ed4ed-6dd1-42f7-b882-e596a253dfe8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '333dc785d4a84d56bd33a6b305dec2f7'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:40:02.258479 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-8b3ed4ed-6dd1-42f7-b882-e596a253dfe8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '333dc785d4a84d56bd33a6b305dec2f7'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:40:02.267095 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-8b3ed4ed-6dd1-42f7-b882-e596a253dfe8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '333dc785d4a84d56bd33a6b305dec2f7'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:40:02.267722 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-8b3ed4ed-6dd1-42f7-b882-e596a253dfe8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '333dc785d4a84d56bd33a6b305dec2f7'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:40:02.280647 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1294/2612] 10.222.0.102 () {58 vars in 1229 bytes} [Thu Sep 1 17:40:02 2022] DELETE /identity/v3/users/333dc785d4a84d56bd33a6b305dec2f7 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:40:02.457531 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998e12d1-2381-4cd3-9a3f-0d2f01794dbc service placement] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:40:02.468706 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998e12d1-2381-4cd3-9a3f-0d2f01794dbc service placement] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:40:02.540183 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2fa628-636c-4a25-a64a-93ae2a8f5f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:40:02.541092 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2fa628-636c-4a25-a64a-93ae2a8f5f60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:40:02.541469 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2fa628-636c-4a25-a64a-93ae2a8f5f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:40:02.541804 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2fa628-636c-4a25-a64a-93ae2a8f5f60 admin admin] PATH_INFO: `/v3/projects/9644529eb8a94aa39e326871bc7b73e1` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:40:02.543108 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2fa628-636c-4a25-a64a-93ae2a8f5f60 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9644529eb8a94aa39e326871bc7b73e1)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:40:02.546166 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2fa628-636c-4a25-a64a-93ae2a8f5f60 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:40:02.618033 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-5c2fa628-636c-4a25-a64a-93ae2a8f5f60 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9644529eb8a94aa39e326871bc7b73e1', 'request_id': 'req-5c2fa628-636c-4a25-a64a-93ae2a8f5f60'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:40:02.630722 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-5c2fa628-636c-4a25-a64a-93ae2a8f5f60 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9644529eb8a94aa39e326871bc7b73e1', 'request_id': 'req-5c2fa628-636c-4a25-a64a-93ae2a8f5f60'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:40:02.638549 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-5c2fa628-636c-4a25-a64a-93ae2a8f5f60 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9644529eb8a94aa39e326871bc7b73e1', 'request_id': 'req-5c2fa628-636c-4a25-a64a-93ae2a8f5f60'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:40:02.652748 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-5c2fa628-636c-4a25-a64a-93ae2a8f5f60 admin admin] The token cache is being invalidate because project 9644529eb8a94aa39e326871bc7b73e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:40:02.653047 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-5c2fa628-636c-4a25-a64a-93ae2a8f5f60 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:40:02.653717 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-5c2fa628-636c-4a25-a64a-93ae2a8f5f60 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:40:02.655390 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1319/2613] 10.222.0.102 () {58 vars in 1238 bytes} [Thu Sep 1 17:40:02 2022] DELETE /identity/v3/projects/9644529eb8a94aa39e326871bc7b73e1 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:40:02.659284 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3ed4ed-6dd1-42f7-b882-e596a253dfe8 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:40:02.660777 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-8b3ed4ed-6dd1-42f7-b882-e596a253dfe8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:40:02.673981 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3ed4ed-6dd1-42f7-b882-e596a253dfe8 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:40:02.735139 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7926f8c4-6857-4a01-aa5c-56c6410c9a8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:40:02.735794 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7926f8c4-6857-4a01-aa5c-56c6410c9a8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:40:02.736073 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7926f8c4-6857-4a01-aa5c-56c6410c9a8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:40:02.736345 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7926f8c4-6857-4a01-aa5c-56c6410c9a8d admin admin] PATH_INFO: `/v3/users/b17a4e748a5f4d3789c3d8617930632a` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:40:02.737520 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7926f8c4-6857-4a01-aa5c-56c6410c9a8d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b17a4e748a5f4d3789c3d8617930632a)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:40:02.739438 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7926f8c4-6857-4a01-aa5c-56c6410c9a8d admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:40:02.787744 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-7926f8c4-6857-4a01-aa5c-56c6410c9a8d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b17a4e748a5f4d3789c3d8617930632a'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:40:02.788512 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-7926f8c4-6857-4a01-aa5c-56c6410c9a8d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b17a4e748a5f4d3789c3d8617930632a'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:40:02.800820 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-7926f8c4-6857-4a01-aa5c-56c6410c9a8d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b17a4e748a5f4d3789c3d8617930632a'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:40:02.809367 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-7926f8c4-6857-4a01-aa5c-56c6410c9a8d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b17a4e748a5f4d3789c3d8617930632a'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:40:02.810238 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-7926f8c4-6857-4a01-aa5c-56c6410c9a8d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b17a4e748a5f4d3789c3d8617930632a'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:40:02.822910 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1295/2614] 10.222.0.102 () {58 vars in 1229 bytes} [Thu Sep 1 17:40:02 2022] DELETE /identity/v3/users/b17a4e748a5f4d3789c3d8617930632a => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:40:03.005635 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2fa628-636c-4a25-a64a-93ae2a8f5f60 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:40:03.007291 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-5c2fa628-636c-4a25-a64a-93ae2a8f5f60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:40:03.025637 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2fa628-636c-4a25-a64a-93ae2a8f5f60 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:40:03.075797 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:40:03.096116 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fbd5c7-33e1-42fc-af99-0195308ab330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:40:03.096768 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fbd5c7-33e1-42fc-af99-0195308ab330 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:40:03.097085 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fbd5c7-33e1-42fc-af99-0195308ab330 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:40:03.097436 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fbd5c7-33e1-42fc-af99-0195308ab330 admin admin] PATH_INFO: `/v3/projects/230a2d829b2e4013a0bb63abd797c373` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:40:03.098643 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fbd5c7-33e1-42fc-af99-0195308ab330 admin admin] RBAC: Authorizing `identity:delete_project(project_id=230a2d829b2e4013a0bb63abd797c373)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:40:03.100682 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fbd5c7-33e1-42fc-af99-0195308ab330 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:40:03.150606 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-53fbd5c7-33e1-42fc-af99-0195308ab330 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '230a2d829b2e4013a0bb63abd797c373', 'request_id': 'req-53fbd5c7-33e1-42fc-af99-0195308ab330'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:40:03.158257 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-53fbd5c7-33e1-42fc-af99-0195308ab330 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '230a2d829b2e4013a0bb63abd797c373', 'request_id': 'req-53fbd5c7-33e1-42fc-af99-0195308ab330'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:40:03.164999 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-53fbd5c7-33e1-42fc-af99-0195308ab330 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '230a2d829b2e4013a0bb63abd797c373', 'request_id': 'req-53fbd5c7-33e1-42fc-af99-0195308ab330'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:40:03.176530 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-53fbd5c7-33e1-42fc-af99-0195308ab330 admin admin] The token cache is being invalidate because project 230a2d829b2e4013a0bb63abd797c373 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:40:03.176907 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-53fbd5c7-33e1-42fc-af99-0195308ab330 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:40:03.177648 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-53fbd5c7-33e1-42fc-af99-0195308ab330 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:40:03.179229 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1320/2615] 10.222.0.102 () {58 vars in 1238 bytes} [Thu Sep 1 17:40:03 2022] DELETE /identity/v3/projects/230a2d829b2e4013a0bb63abd797c373 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:40:09.224996 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bc9854-6596-46a0-9bc2-8b54db610a58 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:40:09.225500 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bc9854-6596-46a0-9bc2-8b54db610a58 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:40:09.225605 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bc9854-6596-46a0-9bc2-8b54db610a58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:40:09.246384 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-61bc9854-6596-46a0-9bc2-8b54db610a58 None None] MFA Rules not processed for user `ae0aab3e503249d49ff9473b4990ff63`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:40:09.247603 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-61bc9854-6596-46a0-9bc2-8b54db610a58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:40:09.268228 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:40:09.309074 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1296/2616] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:40:09 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:40:09.615465 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fbd5c7-33e1-42fc-af99-0195308ab330 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:40:09.616893 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-53fbd5c7-33e1-42fc-af99-0195308ab330 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:40:09.672837 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fbd5c7-33e1-42fc-af99-0195308ab330 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:40:09.676883 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50174195-663a-4444-812e-5ca80c63f212 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:40:09.677514 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50174195-663a-4444-812e-5ca80c63f212 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:40:09.677793 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50174195-663a-4444-812e-5ca80c63f212 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:40:09.678066 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50174195-663a-4444-812e-5ca80c63f212 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:40:09.679744 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-50174195-663a-4444-812e-5ca80c63f212 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:40:09.688892 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50174195-663a-4444-812e-5ca80c63f212 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:40:09.690346 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50174195-663a-4444-812e-5ca80c63f212 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:40:09.698844 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1321/2617] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:40:09 2022] GET /identity/v3/auth/tokens => generated 3131 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:40:13.025334 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9514a29-e0c9-4cc4-840f-45cc74c41201 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:40:13.025813 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9514a29-e0c9-4cc4-840f-45cc74c41201 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:40:13.025926 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9514a29-e0c9-4cc4-840f-45cc74c41201 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:40:13.045883 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-a9514a29-e0c9-4cc4-840f-45cc74c41201 None None] MFA Rules not processed for user `575a76b99ea945efb1024fbb3dc81aaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:40:13.047110 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-a9514a29-e0c9-4cc4-840f-45cc74c41201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:40:13.108250 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1297/2618] 10.222.0.102 () {58 vars in 1014 bytes} [Thu Sep 1 17:40:13 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:40:13.121223 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50174195-663a-4444-812e-5ca80c63f212 service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:40:13.122714 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-50174195-663a-4444-812e-5ca80c63f212 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:40:13.192617 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50174195-663a-4444-812e-5ca80c63f212 service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:40:13.197932 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad9cd10-385a-4236-af2d-ce9d16aeda2a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd55edf6cd1804006aa8269b4c6f80aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:40:13.198545 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad9cd10-385a-4236-af2d-ce9d16aeda2a service swift] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:40:13.198825 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad9cd10-385a-4236-af2d-ce9d16aeda2a service swift] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:40:13.199100 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad9cd10-385a-4236-af2d-ce9d16aeda2a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:40:13.207533 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad9cd10-385a-4236-af2d-ce9d16aeda2a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:40:13.209122 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad9cd10-385a-4236-af2d-ce9d16aeda2a service swift] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:40:13.217257 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1322/2619] 10.222.0.102 () {60 vars in 1389 bytes} [Thu Sep 1 17:40:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 17:40:13.788300 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6170f6d4-fff0-4441-b388-0a2de4ab0f24 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:40:13.788584 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6170f6d4-fff0-4441-b388-0a2de4ab0f24 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:40:13.788852 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6170f6d4-fff0-4441-b388-0a2de4ab0f24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:40:13.808376 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-6170f6d4-fff0-4441-b388-0a2de4ab0f24 None None] MFA Rules not processed for user `575a76b99ea945efb1024fbb3dc81aaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:40:13.809617 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-6170f6d4-fff0-4441-b388-0a2de4ab0f24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:40:13.817061 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1298/2620] 10.222.0.102 () {58 vars in 972 bytes} [Thu Sep 1 17:40:13 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:40:13.830052 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad9cd10-385a-4236-af2d-ce9d16aeda2a service swift] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:40:13.841651 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad9cd10-385a-4236-af2d-ce9d16aeda2a service swift] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:40:13.846838 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c35f44-eba4-4dc8-89a8-9b9bf1400a47 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd55edf6cd1804006aa8269b4c6f80aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:40:13.848197 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c35f44-eba4-4dc8-89a8-9b9bf1400a47 service swift] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:40:13.848729 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c35f44-eba4-4dc8-89a8-9b9bf1400a47 service swift] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:40:13.849078 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c35f44-eba4-4dc8-89a8-9b9bf1400a47 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:40:13.860086 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c35f44-eba4-4dc8-89a8-9b9bf1400a47 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:40:13.861919 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c35f44-eba4-4dc8-89a8-9b9bf1400a47 service swift] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:40:13.865600 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:40:13.879447 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1323/2621] 10.222.0.102 () {60 vars in 1389 bytes} [Thu Sep 1 17:40:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 17:40:14.097413 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849f6997-d9d3-4e7a-86dd-bd7a12524675 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:40:14.097876 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849f6997-d9d3-4e7a-86dd-bd7a12524675 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:40:14.097967 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849f6997-d9d3-4e7a-86dd-bd7a12524675 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:40:14.116673 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-849f6997-d9d3-4e7a-86dd-bd7a12524675 None None] MFA Rules not processed for user `575a76b99ea945efb1024fbb3dc81aaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:40:14.117901 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-849f6997-d9d3-4e7a-86dd-bd7a12524675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:40:14.124929 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1299/2622] 10.222.0.102 () {58 vars in 1014 bytes} [Thu Sep 1 17:40:14 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:40:14.137058 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c35f44-eba4-4dc8-89a8-9b9bf1400a47 service swift] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:40:14.148644 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c35f44-eba4-4dc8-89a8-9b9bf1400a47 service swift] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:40:14.153588 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9032d6-39a7-45de-a6be-059c3fdd7e24 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd55edf6cd1804006aa8269b4c6f80aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:40:14.154209 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9032d6-39a7-45de-a6be-059c3fdd7e24 service swift] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:40:14.154488 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9032d6-39a7-45de-a6be-059c3fdd7e24 service swift] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:40:14.154759 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9032d6-39a7-45de-a6be-059c3fdd7e24 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:40:14.162717 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9032d6-39a7-45de-a6be-059c3fdd7e24 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:40:14.164288 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9032d6-39a7-45de-a6be-059c3fdd7e24 service swift] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:40:14.172695 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1324/2623] 10.222.0.102 () {60 vars in 1389 bytes} [Thu Sep 1 17:40:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 17:40:14.484820 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b35db18-d543-4722-8c77-84b9e68836b1 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:40:14.485115 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b35db18-d543-4722-8c77-84b9e68836b1 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:40:14.485422 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b35db18-d543-4722-8c77-84b9e68836b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:40:14.506716 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-9b35db18-d543-4722-8c77-84b9e68836b1 None None] MFA Rules not processed for user `575a76b99ea945efb1024fbb3dc81aaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:40:14.507922 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-9b35db18-d543-4722-8c77-84b9e68836b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:40:14.515105 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1300/2624] 10.222.0.102 () {58 vars in 972 bytes} [Thu Sep 1 17:40:14 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:40:14.527208 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9032d6-39a7-45de-a6be-059c3fdd7e24 service swift] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:40:14.539226 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9032d6-39a7-45de-a6be-059c3fdd7e24 service swift] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:40:14.543926 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db116cd3-69c6-468e-8bc8-517047fe07ea service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd55edf6cd1804006aa8269b4c6f80aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:40:14.544548 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db116cd3-69c6-468e-8bc8-517047fe07ea service swift] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:40:14.544825 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db116cd3-69c6-468e-8bc8-517047fe07ea service swift] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:40:14.545096 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db116cd3-69c6-468e-8bc8-517047fe07ea service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:40:14.553989 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db116cd3-69c6-468e-8bc8-517047fe07ea service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:40:14.555580 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db116cd3-69c6-468e-8bc8-517047fe07ea service swift] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:40:14.563783 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1325/2625] 10.222.0.102 () {60 vars in 1389 bytes} [Thu Sep 1 17:40:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 17:40:16.629095 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b35db18-d543-4722-8c77-84b9e68836b1 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:40:16.630526 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-9b35db18-d543-4722-8c77-84b9e68836b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:40:16.696427 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b35db18-d543-4722-8c77-84b9e68836b1 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:40:16.701648 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7cd5f0-d803-467f-99e0-c31e1ba7bb5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:40:16.702445 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7cd5f0-d803-467f-99e0-c31e1ba7bb5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:40:16.702738 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7cd5f0-d803-467f-99e0-c31e1ba7bb5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:40:16.703012 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7cd5f0-d803-467f-99e0-c31e1ba7bb5a admin admin] PATH_INFO: `/v3/users/44f3a9807c30494f93e52147967ddb3a` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:40:16.704148 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7cd5f0-d803-467f-99e0-c31e1ba7bb5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=44f3a9807c30494f93e52147967ddb3a)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:40:16.706088 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7cd5f0-d803-467f-99e0-c31e1ba7bb5a admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:40:16.754423 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-4a7cd5f0-d803-467f-99e0-c31e1ba7bb5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44f3a9807c30494f93e52147967ddb3a'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:40:16.755087 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-4a7cd5f0-d803-467f-99e0-c31e1ba7bb5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44f3a9807c30494f93e52147967ddb3a'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:40:16.760525 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:40:16.769482 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-4a7cd5f0-d803-467f-99e0-c31e1ba7bb5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44f3a9807c30494f93e52147967ddb3a'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:40:16.778054 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-4a7cd5f0-d803-467f-99e0-c31e1ba7bb5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44f3a9807c30494f93e52147967ddb3a'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:40:16.778679 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-4a7cd5f0-d803-467f-99e0-c31e1ba7bb5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44f3a9807c30494f93e52147967ddb3a'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:40:16.791538 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1301/2626] 10.222.0.102 () {58 vars in 1229 bytes} [Thu Sep 1 17:40:16 2022] DELETE /identity/v3/users/44f3a9807c30494f93e52147967ddb3a => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:40:16.795942 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05542391-bef4-46e2-a849-7d8b26e5ee73 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:40:16.796455 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05542391-bef4-46e2-a849-7d8b26e5ee73 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:40:16.796676 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05542391-bef4-46e2-a849-7d8b26e5ee73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:40:16.819097 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-05542391-bef4-46e2-a849-7d8b26e5ee73 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:40:16.820317 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-05542391-bef4-46e2-a849-7d8b26e5ee73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:40:16.898292 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1326/2627] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:40:16 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:40:16.907603 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7cd5f0-d803-467f-99e0-c31e1ba7bb5a admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:40:16.909077 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-4a7cd5f0-d803-467f-99e0-c31e1ba7bb5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:40:16.976252 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7cd5f0-d803-467f-99e0-c31e1ba7bb5a admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:40:16.982474 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d3b464-68d1-4621-ac85-b6ed56a9dc14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:40:16.983188 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d3b464-68d1-4621-ac85-b6ed56a9dc14 service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:40:16.983568 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d3b464-68d1-4621-ac85-b6ed56a9dc14 service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:40:16.983939 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d3b464-68d1-4621-ac85-b6ed56a9dc14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:40:16.993973 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d3b464-68d1-4621-ac85-b6ed56a9dc14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:40:16.995501 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d3b464-68d1-4621-ac85-b6ed56a9dc14 service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:40:17.004982 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1302/2628] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:40:16 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 17:40:17.323074 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05542391-bef4-46e2-a849-7d8b26e5ee73 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:40:17.325032 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-05542391-bef4-46e2-a849-7d8b26e5ee73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:40:17.340152 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05542391-bef4-46e2-a849-7d8b26e5ee73 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:40:17.345239 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685451e3-f58f-426c-adfb-8af92d296da1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:40:17.345982 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685451e3-f58f-426c-adfb-8af92d296da1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:40:17.346471 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685451e3-f58f-426c-adfb-8af92d296da1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:40:17.347564 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685451e3-f58f-426c-adfb-8af92d296da1 admin admin] PATH_INFO: `/v3/projects/fd4ba94e5d9a4bf185bbbdf5e4c84ce8` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:40:17.348626 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685451e3-f58f-426c-adfb-8af92d296da1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd4ba94e5d9a4bf185bbbdf5e4c84ce8)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:40:17.350679 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685451e3-f58f-426c-adfb-8af92d296da1 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:40:17.397093 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-685451e3-f58f-426c-adfb-8af92d296da1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd4ba94e5d9a4bf185bbbdf5e4c84ce8', 'request_id': 'req-685451e3-f58f-426c-adfb-8af92d296da1'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:40:17.404102 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-685451e3-f58f-426c-adfb-8af92d296da1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd4ba94e5d9a4bf185bbbdf5e4c84ce8', 'request_id': 'req-685451e3-f58f-426c-adfb-8af92d296da1'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:40:17.410844 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-685451e3-f58f-426c-adfb-8af92d296da1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd4ba94e5d9a4bf185bbbdf5e4c84ce8', 'request_id': 'req-685451e3-f58f-426c-adfb-8af92d296da1'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:40:17.421001 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-685451e3-f58f-426c-adfb-8af92d296da1 admin admin] The token cache is being invalidate because project fd4ba94e5d9a4bf185bbbdf5e4c84ce8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:40:17.421296 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-685451e3-f58f-426c-adfb-8af92d296da1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:40:17.421948 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-685451e3-f58f-426c-adfb-8af92d296da1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:40:17.423409 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1327/2629] 10.222.0.102 () {58 vars in 1238 bytes} [Thu Sep 1 17:40:17 2022] DELETE /identity/v3/projects/fd4ba94e5d9a4bf185bbbdf5e4c84ce8 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:40:17.428424 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db22bd4a-cd1a-4fad-ac3d-82ea3ba3b767 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:40:17.428731 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db22bd4a-cd1a-4fad-ac3d-82ea3ba3b767 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:40:17.429005 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db22bd4a-cd1a-4fad-ac3d-82ea3ba3b767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:40:17.448027 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-db22bd4a-cd1a-4fad-ac3d-82ea3ba3b767 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:40:17.449208 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-db22bd4a-cd1a-4fad-ac3d-82ea3ba3b767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:40:17.507997 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1303/2630] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:40:17 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:40:17.515674 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a5843e-1003-4bf1-bf62-65ecdd710e2f None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:40:17.515966 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a5843e-1003-4bf1-bf62-65ecdd710e2f None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:40:17.516248 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a5843e-1003-4bf1-bf62-65ecdd710e2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:40:17.537074 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-06a5843e-1003-4bf1-bf62-65ecdd710e2f None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:40:17.538360 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-06a5843e-1003-4bf1-bf62-65ecdd710e2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:40:17.546842 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1328/2631] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:40:17 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:40:17.550973 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db22bd4a-cd1a-4fad-ac3d-82ea3ba3b767 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:40:17.560589 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db22bd4a-cd1a-4fad-ac3d-82ea3ba3b767 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:40:17.565176 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6811bee2-f341-4336-b1c7-f25ab375d6d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:40:17.565797 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6811bee2-f341-4336-b1c7-f25ab375d6d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:40:17.566078 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6811bee2-f341-4336-b1c7-f25ab375d6d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:40:17.566352 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6811bee2-f341-4336-b1c7-f25ab375d6d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:40:17.567001 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6811bee2-f341-4336-b1c7-f25ab375d6d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:40:17.567308 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6811bee2-f341-4336-b1c7-f25ab375d6d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:40:17.568887 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6811bee2-f341-4336-b1c7-f25ab375d6d2 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:40:17.580607 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1304/2632] 10.222.0.102 () {58 vars in 1158 bytes} [Thu Sep 1 17:40:17 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:40:17.585183 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a5843e-1003-4bf1-bf62-65ecdd710e2f None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:40:17.594333 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a5843e-1003-4bf1-bf62-65ecdd710e2f None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:40:17.599527 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eda3e49-7604-4d3a-af78-89aaca579a2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:40:17.600106 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eda3e49-7604-4d3a-af78-89aaca579a2c admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:40:17.600382 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eda3e49-7604-4d3a-af78-89aaca579a2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:40:17.600652 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eda3e49-7604-4d3a-af78-89aaca579a2c admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:40:17.601274 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eda3e49-7604-4d3a-af78-89aaca579a2c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:40:17.603219 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eda3e49-7604-4d3a-af78-89aaca579a2c admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:40:17.630342 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1329/2633] 10.222.0.102 () {60 vars in 1158 bytes} [Thu Sep 1 17:40:17 2022] POST /identity/v3/projects => generated 384 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:40:17.634779 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6811bee2-f341-4336-b1c7-f25ab375d6d2 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:40:17.645649 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6811bee2-f341-4336-b1c7-f25ab375d6d2 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:40:17.700323 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e086709-00d1-46f0-a57c-b5f4227190ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:40:17.700912 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e086709-00d1-46f0-a57c-b5f4227190ea admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:40:17.701192 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e086709-00d1-46f0-a57c-b5f4227190ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:40:17.701495 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e086709-00d1-46f0-a57c-b5f4227190ea admin admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:40:17.702121 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e086709-00d1-46f0-a57c-b5f4227190ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:40:17.704054 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e086709-00d1-46f0-a57c-b5f4227190ea admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:40:17.713443 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:40:17.729009 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1305/2634] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:40:17 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:40:17.734383 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eda3e49-7604-4d3a-af78-89aaca579a2c admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:40:17.744820 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eda3e49-7604-4d3a-af78-89aaca579a2c admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:40:17.749763 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05b32b3-d2c3-424c-b468-f3843c67e0c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:40:17.750362 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05b32b3-d2c3-424c-b468-f3843c67e0c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:40:17.750647 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05b32b3-d2c3-424c-b468-f3843c67e0c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:40:17.750924 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05b32b3-d2c3-424c-b468-f3843c67e0c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:40:17.751571 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05b32b3-d2c3-424c-b468-f3843c67e0c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:40:17.753716 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05b32b3-d2c3-424c-b468-f3843c67e0c0 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:40:17.763730 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1330/2635] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:40:17 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:40:17.767511 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e086709-00d1-46f0-a57c-b5f4227190ea admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:40:17.777829 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e086709-00d1-46f0-a57c-b5f4227190ea admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:40:17.782465 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd0e768-1ebb-4ace-8c12-1c230e2a6f8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:40:17.783050 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd0e768-1ebb-4ace-8c12-1c230e2a6f8b admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:40:17.783330 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd0e768-1ebb-4ace-8c12-1c230e2a6f8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:40:17.783603 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd0e768-1ebb-4ace-8c12-1c230e2a6f8b admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:40:17.784240 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd0e768-1ebb-4ace-8c12-1c230e2a6f8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:40:17.785875 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd0e768-1ebb-4ace-8c12-1c230e2a6f8b admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:40:17.796312 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1306/2636] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:40:17 2022] GET /identity/v3/roles => generated 1427 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:40:17.799933 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05b32b3-d2c3-424c-b468-f3843c67e0c0 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:40:17.809614 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05b32b3-d2c3-424c-b468-f3843c67e0c0 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:40:17.814203 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa621054-3c9f-4cf8-9ca3-58a90f2eeca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:40:17.814847 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa621054-3c9f-4cf8-9ca3-58a90f2eeca3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:40:17.815128 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa621054-3c9f-4cf8-9ca3-58a90f2eeca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:40:17.815409 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa621054-3c9f-4cf8-9ca3-58a90f2eeca3 admin admin] PATH_INFO: `/v3/projects/41b4d9513f204230a8ff27614eb1a642/users/d09949305dc04d658d20ec90798aff3c/roles/b75cb360925e4ba5a65f14965c103ffd` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:40:17.828444 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:40:17.832318 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa621054-3c9f-4cf8-9ca3-58a90f2eeca3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41b4d9513f204230a8ff27614eb1a642, user_id=d09949305dc04d658d20ec90798aff3c, role_id=b75cb360925e4ba5a65f14965c103ffd)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:40:17.835303 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa621054-3c9f-4cf8-9ca3-58a90f2eeca3 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:40:17.849697 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1331/2637] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:40:17 2022] PUT /identity/v3/projects/41b4d9513f204230a8ff27614eb1a642/users/d09949305dc04d658d20ec90798aff3c/roles/b75cb360925e4ba5a65f14965c103ffd => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:40:17.854577 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37a2375-7d79-42f1-acb7-3f5c731f45bf None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:40:17.854866 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37a2375-7d79-42f1-acb7-3f5c731f45bf None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:40:17.855164 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37a2375-7d79-42f1-acb7-3f5c731f45bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:40:17.873873 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-a37a2375-7d79-42f1-acb7-3f5c731f45bf None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:40:17.875078 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-a37a2375-7d79-42f1-acb7-3f5c731f45bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:40:17.933200 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1307/2638] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:40:17 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:40:17.943168 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa621054-3c9f-4cf8-9ca3-58a90f2eeca3 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:40:17.944851 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-fa621054-3c9f-4cf8-9ca3-58a90f2eeca3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:40:18.005274 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa621054-3c9f-4cf8-9ca3-58a90f2eeca3 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:40:18.008964 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9078f415-53e6-4253-9639-499cafc6c032 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:40:18.009593 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9078f415-53e6-4253-9639-499cafc6c032 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:40:18.009877 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9078f415-53e6-4253-9639-499cafc6c032 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:40:18.010150 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9078f415-53e6-4253-9639-499cafc6c032 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:40:18.019291 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9078f415-53e6-4253-9639-499cafc6c032 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:40:18.020717 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9078f415-53e6-4253-9639-499cafc6c032 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:40:18.030265 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1332/2639] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:40:17 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:40:18.631802 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5f0320-dae9-40b9-8eb6-928edf4ef25c None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:40:18.632091 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5f0320-dae9-40b9-8eb6-928edf4ef25c None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:40:18.632369 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5f0320-dae9-40b9-8eb6-928edf4ef25c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:40:18.661239 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-fd5f0320-dae9-40b9-8eb6-928edf4ef25c None None] MFA Rules not processed for user `d09949305dc04d658d20ec90798aff3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:40:18.662481 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-fd5f0320-dae9-40b9-8eb6-928edf4ef25c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:40:18.731064 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1308/2640] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:40:18 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:40:18.742410 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9078f415-53e6-4253-9639-499cafc6c032 service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:40:18.744154 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-9078f415-53e6-4253-9639-499cafc6c032 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:40:18.805581 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9078f415-53e6-4253-9639-499cafc6c032 service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:40:18.814742 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd77453-6de4-4a53-a423-54e8ebcb58a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '523ed589872a45a09b6a7675a7fb0b75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:40:18.814742 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd77453-6de4-4a53-a423-54e8ebcb58a3 service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:40:18.815055 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd77453-6de4-4a53-a423-54e8ebcb58a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:40:18.815277 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd77453-6de4-4a53-a423-54e8ebcb58a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:40:18.824783 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd77453-6de4-4a53-a423-54e8ebcb58a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:40:18.826271 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd77453-6de4-4a53-a423-54e8ebcb58a3 service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:40:18.836561 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1333/2641] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:40:18 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:40:20.230768 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f89faf-c63d-4f79-b153-8d726987a9d2 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:40:20.231245 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f89faf-c63d-4f79-b153-8d726987a9d2 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:40:20.231361 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f89faf-c63d-4f79-b153-8d726987a9d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:40:20.251355 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-12f89faf-c63d-4f79-b153-8d726987a9d2 None None] MFA Rules not processed for user `d09949305dc04d658d20ec90798aff3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:40:20.252549 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-12f89faf-c63d-4f79-b153-8d726987a9d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:40:20.259481 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1309/2642] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:40:20 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:40:28.832049 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=55,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:40:30.246484 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:INSERT=2,keystone:SELECT=69 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:40:35.549892 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd77453-6de4-4a53-a423-54e8ebcb58a3 service cinder] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:40:35.551287 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-9bd77453-6de4-4a53-a423-54e8ebcb58a3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:40:35.562686 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd77453-6de4-4a53-a423-54e8ebcb58a3 service cinder] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:40:35.566956 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79289ed3-b994-46a9-923c-89ab99b51f54 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '523ed589872a45a09b6a7675a7fb0b75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:40:35.567547 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79289ed3-b994-46a9-923c-89ab99b51f54 service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:40:35.567828 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79289ed3-b994-46a9-923c-89ab99b51f54 service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:40:35.568092 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79289ed3-b994-46a9-923c-89ab99b51f54 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:40:35.569753 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-79289ed3-b994-46a9-923c-89ab99b51f54 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:40:35.627713 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79289ed3-b994-46a9-923c-89ab99b51f54 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:40:35.629127 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79289ed3-b994-46a9-923c-89ab99b51f54 service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:40:35.638130 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1334/2643] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:40:35 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:40:45.634140 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=28 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:40:48.880605 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3889c6-bb56-4464-8a0c-1458a632b4bc None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:40:48.881150 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3889c6-bb56-4464-8a0c-1458a632b4bc None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:40:48.881150 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3889c6-bb56-4464-8a0c-1458a632b4bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:40:48.901058 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-ab3889c6-bb56-4464-8a0c-1458a632b4bc None None] MFA Rules not processed for user `8f59383e2f6c4da8b683d718f62d25fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:40:48.902305 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-ab3889c6-bb56-4464-8a0c-1458a632b4bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:40:48.909678 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1310/2644] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:40:48 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:40:58.895905 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=3 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:41:11.352779 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79289ed3-b994-46a9-923c-89ab99b51f54 service cinder] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:41:11.362160 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79289ed3-b994-46a9-923c-89ab99b51f54 service cinder] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:41:11.365815 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98454983-7eae-4e7c-81cf-3ca6d1edc9d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:41:11.366422 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98454983-7eae-4e7c-81cf-3ca6d1edc9d4 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:41:11.366702 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98454983-7eae-4e7c-81cf-3ca6d1edc9d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:41:11.366973 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98454983-7eae-4e7c-81cf-3ca6d1edc9d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:41:11.369031 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-98454983-7eae-4e7c-81cf-3ca6d1edc9d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:41:11.591099 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98454983-7eae-4e7c-81cf-3ca6d1edc9d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:41:11.592861 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98454983-7eae-4e7c-81cf-3ca6d1edc9d4 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:41:11.616035 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1335/2645] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:41:11 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:41:11.621706 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3889c6-bb56-4464-8a0c-1458a632b4bc None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:41:11.635629 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3889c6-bb56-4464-8a0c-1458a632b4bc None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:41:11.639451 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d196b3b-2c3f-4efe-9aad-65e468596e88 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '523ed589872a45a09b6a7675a7fb0b75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:41:11.640126 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d196b3b-2c3f-4efe-9aad-65e468596e88 service cinder] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:41:11.640483 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d196b3b-2c3f-4efe-9aad-65e468596e88 service cinder] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:41:11.640864 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d196b3b-2c3f-4efe-9aad-65e468596e88 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:41:11.642926 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-0d196b3b-2c3f-4efe-9aad-65e468596e88 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:41:11.657845 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d196b3b-2c3f-4efe-9aad-65e468596e88 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:41:11.659375 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d196b3b-2c3f-4efe-9aad-65e468596e88 service cinder] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:41:11.675303 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1311/2646] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:41:11 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:41:13.100932 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98454983-7eae-4e7c-81cf-3ca6d1edc9d4 service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:41:13.102381 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-98454983-7eae-4e7c-81cf-3ca6d1edc9d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:41:13.113401 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98454983-7eae-4e7c-81cf-3ca6d1edc9d4 service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:41:13.116953 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99442a40-eae2-4965-8f8a-c660898bea33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:41:13.117571 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99442a40-eae2-4965-8f8a-c660898bea33 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:41:13.117845 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99442a40-eae2-4965-8f8a-c660898bea33 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:41:13.118112 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99442a40-eae2-4965-8f8a-c660898bea33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:41:13.119719 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-99442a40-eae2-4965-8f8a-c660898bea33 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:41:13.182452 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99442a40-eae2-4965-8f8a-c660898bea33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:41:13.183824 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99442a40-eae2-4965-8f8a-c660898bea33 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:41:13.193956 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1336/2647] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:41:13 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:41:16.232836 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968ec32d-a426-4a74-850a-c1ca1d42506d None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:41:16.233383 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968ec32d-a426-4a74-850a-c1ca1d42506d None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:41:16.233483 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968ec32d-a426-4a74-850a-c1ca1d42506d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:41:16.252095 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-968ec32d-a426-4a74-850a-c1ca1d42506d None None] MFA Rules not processed for user `575a76b99ea945efb1024fbb3dc81aaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:41:16.253278 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-968ec32d-a426-4a74-850a-c1ca1d42506d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:41:16.306401 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1312/2648] 10.222.0.102 () {58 vars in 1014 bytes} [Thu Sep 1 17:41:16 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:41:16.352409 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99442a40-eae2-4965-8f8a-c660898bea33 service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:41:16.353876 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-99442a40-eae2-4965-8f8a-c660898bea33 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:41:16.418664 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:41:16.425532 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99442a40-eae2-4965-8f8a-c660898bea33 service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:41:16.430524 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993f2e95-c738-483f-bcf8-b2ddf785ebb5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd55edf6cd1804006aa8269b4c6f80aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:41:16.431126 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993f2e95-c738-483f-bcf8-b2ddf785ebb5 service swift] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:41:16.431405 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993f2e95-c738-483f-bcf8-b2ddf785ebb5 service swift] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:41:16.431676 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993f2e95-c738-483f-bcf8-b2ddf785ebb5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:41:16.440256 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993f2e95-c738-483f-bcf8-b2ddf785ebb5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:41:16.441882 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993f2e95-c738-483f-bcf8-b2ddf785ebb5 service swift] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:41:16.450008 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1337/2649] 10.222.0.102 () {60 vars in 1389 bytes} [Thu Sep 1 17:41:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 17:41:18.892112 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968ec32d-a426-4a74-850a-c1ca1d42506d None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:41:18.893716 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-968ec32d-a426-4a74-850a-c1ca1d42506d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:41:18.911369 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968ec32d-a426-4a74-850a-c1ca1d42506d None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:41:18.917081 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733f1b70-b370-417e-8820-fa69ac236a5b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f8293316baf4774b62fc11d55d50865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:41:18.917802 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733f1b70-b370-417e-8820-fa69ac236a5b service nova] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:41:18.918142 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733f1b70-b370-417e-8820-fa69ac236a5b service nova] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:41:18.918477 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733f1b70-b370-417e-8820-fa69ac236a5b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:41:18.920909 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-733f1b70-b370-417e-8820-fa69ac236a5b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:41:19.010679 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733f1b70-b370-417e-8820-fa69ac236a5b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:41:19.012255 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733f1b70-b370-417e-8820-fa69ac236a5b service nova] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:41:19.038897 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1313/2650] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:41:18 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:41:26.446553 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=5 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:41:29.034346 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=72 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:41:41.265195 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e114b51f-2ddc-4951-a2a9-c05c56df7b0f None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:41:41.265851 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e114b51f-2ddc-4951-a2a9-c05c56df7b0f None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:41:41.266169 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e114b51f-2ddc-4951-a2a9-c05c56df7b0f None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:41:41.267767 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1338/2651] 10.222.0.102 () {54 vars in 926 bytes} [Thu Sep 1 17:41:41 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:41:41.274018 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5fc814-2d67-49b4-92c6-fbb48e9ec4c5 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:41:41.274530 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5fc814-2d67-49b4-92c6-fbb48e9ec4c5 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:41:41.274627 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5fc814-2d67-49b4-92c6-fbb48e9ec4c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:41:41.317825 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-7a5fc814-2d67-49b4-92c6-fbb48e9ec4c5 None None] MFA Rules not processed for user `5f8293316baf4774b62fc11d55d50865`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:41:41.319082 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-7a5fc814-2d67-49b4-92c6-fbb48e9ec4c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:41:41.333365 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1314/2652] 10.222.0.102 () {58 vars in 1023 bytes} [Thu Sep 1 17:41:41 2022] POST /identity/v3/auth/tokens => generated 3382 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:41:41.350692 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e114b51f-2ddc-4951-a2a9-c05c56df7b0f None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:41:41.352423 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-e114b51f-2ddc-4951-a2a9-c05c56df7b0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:41:41.411952 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e114b51f-2ddc-4951-a2a9-c05c56df7b0f None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:41:41.415453 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5417b729-5faa-41b5-addd-7638b01dc542 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '039683b8d1d146c6add5ffa235d06f91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:41:41.416106 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5417b729-5faa-41b5-addd-7638b01dc542 service glance] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:41:41.416387 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5417b729-5faa-41b5-addd-7638b01dc542 service glance] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:41:41.416794 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5417b729-5faa-41b5-addd-7638b01dc542 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:41:41.427366 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5417b729-5faa-41b5-addd-7638b01dc542 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:41:41.428779 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5417b729-5faa-41b5-addd-7638b01dc542 service glance] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:41:41.439201 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1339/2653] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:41:41 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:41:41.506639 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee90d6f7-79c5-4c38-867c-3b5aef744d1f None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:41:41.506928 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee90d6f7-79c5-4c38-867c-3b5aef744d1f None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:41:41.507230 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee90d6f7-79c5-4c38-867c-3b5aef744d1f None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:41:41.508600 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1315/2654] 10.222.0.102 () {54 vars in 926 bytes} [Thu Sep 1 17:41:41 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:41:41.513800 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5588163-73a9-4102-97da-7cca654b7575 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:41:41.514091 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5588163-73a9-4102-97da-7cca654b7575 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:41:41.514367 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5588163-73a9-4102-97da-7cca654b7575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:41:41.554398 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-a5588163-73a9-4102-97da-7cca654b7575 None None] MFA Rules not processed for user `7c7d3594fdab4191be1d841af43a5c79`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:41:41.555613 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-a5588163-73a9-4102-97da-7cca654b7575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:41:41.568634 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1340/2655] 10.222.0.102 () {58 vars in 1023 bytes} [Thu Sep 1 17:41:41 2022] POST /identity/v3/auth/tokens => generated 3131 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:41:41.580737 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee90d6f7-79c5-4c38-867c-3b5aef744d1f None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:41:41.594276 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee90d6f7-79c5-4c38-867c-3b5aef744d1f None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:41:41.597752 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b14d09e-62c3-4222-9a95-e04ffab5b800 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:41:41.598332 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b14d09e-62c3-4222-9a95-e04ffab5b800 service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:41:41.598604 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b14d09e-62c3-4222-9a95-e04ffab5b800 service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:41:41.598872 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b14d09e-62c3-4222-9a95-e04ffab5b800 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:41:41.606243 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b14d09e-62c3-4222-9a95-e04ffab5b800 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:41:41.607652 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b14d09e-62c3-4222-9a95-e04ffab5b800 service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:41:41.615087 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1316/2656] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:41:41 2022] GET /identity/v3/auth/tokens => generated 3131 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:41:41.696286 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358352fe-e1d3-4fc6-b8ef-74c85cbdfd9e None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:41:41.696569 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358352fe-e1d3-4fc6-b8ef-74c85cbdfd9e None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:41:41.696838 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358352fe-e1d3-4fc6-b8ef-74c85cbdfd9e None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:41:41.698208 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1341/2657] 10.222.0.102 () {54 vars in 926 bytes} [Thu Sep 1 17:41:41 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:41:41.703916 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a474100b-5e31-4b87-bcdd-977a7b2a8d8f None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:41:41.704202 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a474100b-5e31-4b87-bcdd-977a7b2a8d8f None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:41:41.704479 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a474100b-5e31-4b87-bcdd-977a7b2a8d8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:41:41.724157 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-a474100b-5e31-4b87-bcdd-977a7b2a8d8f None None] MFA Rules not processed for user `5f8293316baf4774b62fc11d55d50865`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:41:41.725429 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-a474100b-5e31-4b87-bcdd-977a7b2a8d8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:41:41.734475 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1317/2658] 10.222.0.102 () {58 vars in 1023 bytes} [Thu Sep 1 17:41:41 2022] POST /identity/v3/auth/tokens => generated 3382 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:41:41.744678 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358352fe-e1d3-4fc6-b8ef-74c85cbdfd9e None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:41:41.755555 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358352fe-e1d3-4fc6-b8ef-74c85cbdfd9e None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:41:41.759046 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb29a21-69f2-4ece-8cfd-0d68474a2904 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:41:41.759676 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb29a21-69f2-4ece-8cfd-0d68474a2904 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:41:41.759949 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb29a21-69f2-4ece-8cfd-0d68474a2904 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:41:41.760308 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb29a21-69f2-4ece-8cfd-0d68474a2904 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:41:41.770604 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb29a21-69f2-4ece-8cfd-0d68474a2904 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:41:41.772013 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb29a21-69f2-4ece-8cfd-0d68474a2904 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:41:41.783028 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1342/2659] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:41:41 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:41:43.072230 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a474100b-5e31-4b87-bcdd-977a7b2a8d8f None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:41:43.082278 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a474100b-5e31-4b87-bcdd-977a7b2a8d8f None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:41:43.085910 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e611f29f-346f-49d7-9ebb-cb547435e14d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:41:43.086508 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e611f29f-346f-49d7-9ebb-cb547435e14d service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:41:43.086788 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e611f29f-346f-49d7-9ebb-cb547435e14d service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:41:43.087059 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e611f29f-346f-49d7-9ebb-cb547435e14d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:41:43.088780 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-e611f29f-346f-49d7-9ebb-cb547435e14d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:41:43.098178 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e611f29f-346f-49d7-9ebb-cb547435e14d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:41:43.099588 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e611f29f-346f-49d7-9ebb-cb547435e14d service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:41:43.108002 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1318/2660] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:41:43 2022] GET /identity/v3/auth/tokens => generated 3131 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:41:44.050089 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9fe1b5-911a-4e00-8bc7-d97b66023edd None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:41:44.050554 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9fe1b5-911a-4e00-8bc7-d97b66023edd None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:41:44.050648 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9fe1b5-911a-4e00-8bc7-d97b66023edd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:41:44.099160 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-5a9fe1b5-911a-4e00-8bc7-d97b66023edd None None] MFA Rules not processed for user `575a76b99ea945efb1024fbb3dc81aaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:41:44.100363 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-5a9fe1b5-911a-4e00-8bc7-d97b66023edd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:41:44.116511 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1343/2661] 10.222.0.102 () {58 vars in 1014 bytes} [Thu Sep 1 17:41:44 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:41:44.128034 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e611f29f-346f-49d7-9ebb-cb547435e14d service neutron] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:41:44.149800 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e611f29f-346f-49d7-9ebb-cb547435e14d service neutron] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:41:44.155424 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51580f9c-13fe-467d-ab4d-91c79cc10767 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd55edf6cd1804006aa8269b4c6f80aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:41:44.156018 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51580f9c-13fe-467d-ab4d-91c79cc10767 service swift] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:41:44.156298 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51580f9c-13fe-467d-ab4d-91c79cc10767 service swift] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:41:44.156572 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51580f9c-13fe-467d-ab4d-91c79cc10767 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:41:44.164549 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51580f9c-13fe-467d-ab4d-91c79cc10767 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:41:44.166139 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51580f9c-13fe-467d-ab4d-91c79cc10767 service swift] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:41:44.173944 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1319/2662] 10.222.0.102 () {60 vars in 1389 bytes} [Thu Sep 1 17:41:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 17:41:54.113815 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=59 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:41:54.170833 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=34 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:42:07.332219 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9fe1b5-911a-4e00-8bc7-d97b66023edd None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:42:07.334086 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-5a9fe1b5-911a-4e00-8bc7-d97b66023edd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:42:07.352352 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9fe1b5-911a-4e00-8bc7-d97b66023edd None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:42:07.357971 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9796b24-c09d-435c-8856-9677b518e112 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f8293316baf4774b62fc11d55d50865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:42:07.358588 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9796b24-c09d-435c-8856-9677b518e112 service nova] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:42:07.358875 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9796b24-c09d-435c-8856-9677b518e112 service nova] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:42:07.359125 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9796b24-c09d-435c-8856-9677b518e112 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:42:07.361961 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-f9796b24-c09d-435c-8856-9677b518e112 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:42:07.374225 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9796b24-c09d-435c-8856-9677b518e112 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:42:07.375981 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9796b24-c09d-435c-8856-9677b518e112 service nova] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:42:07.387526 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1344/2663] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:42:07 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:42:10.192756 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0936775d-7c36-42b8-b513-5cdbadb6e1b7 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:42:10.193325 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0936775d-7c36-42b8-b513-5cdbadb6e1b7 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:42:10.193325 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0936775d-7c36-42b8-b513-5cdbadb6e1b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:42:10.213612 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-0936775d-7c36-42b8-b513-5cdbadb6e1b7 None None] MFA Rules not processed for user `575a76b99ea945efb1024fbb3dc81aaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:42:10.214991 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-0936775d-7c36-42b8-b513-5cdbadb6e1b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:42:10.222545 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1320/2664] 10.222.0.102 () {58 vars in 972 bytes} [Thu Sep 1 17:42:10 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:42:10.234559 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9796b24-c09d-435c-8856-9677b518e112 service nova] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:42:10.246375 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9796b24-c09d-435c-8856-9677b518e112 service nova] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:42:10.251748 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc921e1-5cc4-4ca1-9ad6-160f40ebabb5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd55edf6cd1804006aa8269b4c6f80aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:42:10.252889 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc921e1-5cc4-4ca1-9ad6-160f40ebabb5 service swift] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:42:10.253186 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc921e1-5cc4-4ca1-9ad6-160f40ebabb5 service swift] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:42:10.253495 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc921e1-5cc4-4ca1-9ad6-160f40ebabb5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:42:10.261871 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc921e1-5cc4-4ca1-9ad6-160f40ebabb5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:42:10.263481 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc921e1-5cc4-4ca1-9ad6-160f40ebabb5 service swift] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:42:10.271980 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1345/2665] 10.222.0.102 () {60 vars in 1389 bytes} [Thu Sep 1 17:42:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 17:42:20.208121 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=3 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:42:20.268318 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=12 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:42:26.012313 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6d9143-43b0-48e4-ba47-f33033e66456 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:42:26.012772 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6d9143-43b0-48e4-ba47-f33033e66456 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:42:26.012867 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6d9143-43b0-48e4-ba47-f33033e66456 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:42:26.031250 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-df6d9143-43b0-48e4-ba47-f33033e66456 None None] MFA Rules not processed for user `d09949305dc04d658d20ec90798aff3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:42:26.032459 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-df6d9143-43b0-48e4-ba47-f33033e66456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:42:26.039491 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1321/2666] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:42:26 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:42:36.026428 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=3 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:43:11.645683 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc921e1-5cc4-4ca1-9ad6-160f40ebabb5 service swift] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:43:11.658211 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc921e1-5cc4-4ca1-9ad6-160f40ebabb5 service swift] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:43:11.663964 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91437d65-531f-412f-9a36-b6057745116c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f8293316baf4774b62fc11d55d50865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:43:11.664701 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91437d65-531f-412f-9a36-b6057745116c service nova] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:43:11.665082 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91437d65-531f-412f-9a36-b6057745116c service nova] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:43:11.665525 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91437d65-531f-412f-9a36-b6057745116c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:43:11.667986 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-91437d65-531f-412f-9a36-b6057745116c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:43:11.681792 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91437d65-531f-412f-9a36-b6057745116c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:43:11.683378 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91437d65-531f-412f-9a36-b6057745116c service nova] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:43:11.694396 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1346/2667] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:43:11 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:43:15.175640 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6d9143-43b0-48e4-ba47-f33033e66456 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:43:15.185070 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6d9143-43b0-48e4-ba47-f33033e66456 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:43:15.188529 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96f9110-dc83-4f64-b0e5-7ec16236bff7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '523ed589872a45a09b6a7675a7fb0b75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:43:15.189116 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96f9110-dc83-4f64-b0e5-7ec16236bff7 service cinder] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:43:15.189427 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96f9110-dc83-4f64-b0e5-7ec16236bff7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:43:15.189696 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96f9110-dc83-4f64-b0e5-7ec16236bff7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:43:15.191378 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-d96f9110-dc83-4f64-b0e5-7ec16236bff7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:43:15.271096 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96f9110-dc83-4f64-b0e5-7ec16236bff7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:43:15.273168 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96f9110-dc83-4f64-b0e5-7ec16236bff7 service cinder] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:43:15.291090 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1322/2668] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:43:15 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:43:21.697553 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=6 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:43:25.288518 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=41 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:43:40.520357 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91437d65-531f-412f-9a36-b6057745116c service nova] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:43:40.531720 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91437d65-531f-412f-9a36-b6057745116c service nova] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:43:40.536198 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99b7c95-c167-40c3-b378-902c62ef47ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:43:40.536815 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99b7c95-c167-40c3-b378-902c62ef47ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:43:40.537091 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99b7c95-c167-40c3-b378-902c62ef47ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:43:40.537389 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99b7c95-c167-40c3-b378-902c62ef47ae admin admin] PATH_INFO: `/v3/users/d09949305dc04d658d20ec90798aff3c` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:43:40.538494 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99b7c95-c167-40c3-b378-902c62ef47ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=d09949305dc04d658d20ec90798aff3c)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:43:40.540423 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99b7c95-c167-40c3-b378-902c62ef47ae admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:43:40.590234 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b99b7c95-c167-40c3-b378-902c62ef47ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd09949305dc04d658d20ec90798aff3c'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:43:40.602089 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b99b7c95-c167-40c3-b378-902c62ef47ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd09949305dc04d658d20ec90798aff3c'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:43:40.609467 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b99b7c95-c167-40c3-b378-902c62ef47ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd09949305dc04d658d20ec90798aff3c'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:43:40.610135 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b99b7c95-c167-40c3-b378-902c62ef47ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd09949305dc04d658d20ec90798aff3c'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:43:40.610792 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b99b7c95-c167-40c3-b378-902c62ef47ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd09949305dc04d658d20ec90798aff3c'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:43:40.623122 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1347/2669] 10.222.0.102 () {58 vars in 1229 bytes} [Thu Sep 1 17:43:40 2022] DELETE /identity/v3/users/d09949305dc04d658d20ec90798aff3c => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:43:40.797101 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96f9110-dc83-4f64-b0e5-7ec16236bff7 service cinder] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:43:40.807738 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96f9110-dc83-4f64-b0e5-7ec16236bff7 service cinder] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:43:40.880358 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f031aa-2aff-4b27-9de7-69071750e4de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:43:40.881038 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f031aa-2aff-4b27-9de7-69071750e4de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:43:40.881327 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f031aa-2aff-4b27-9de7-69071750e4de admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:43:40.881615 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f031aa-2aff-4b27-9de7-69071750e4de admin admin] PATH_INFO: `/v3/projects/41b4d9513f204230a8ff27614eb1a642` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:43:40.882721 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f031aa-2aff-4b27-9de7-69071750e4de admin admin] RBAC: Authorizing `identity:delete_project(project_id=41b4d9513f204230a8ff27614eb1a642)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:43:40.884660 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f031aa-2aff-4b27-9de7-69071750e4de admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:43:40.934354 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-81f031aa-2aff-4b27-9de7-69071750e4de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41b4d9513f204230a8ff27614eb1a642', 'request_id': 'req-81f031aa-2aff-4b27-9de7-69071750e4de'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:43:40.940188 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-81f031aa-2aff-4b27-9de7-69071750e4de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41b4d9513f204230a8ff27614eb1a642', 'request_id': 'req-81f031aa-2aff-4b27-9de7-69071750e4de'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:43:40.949000 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-81f031aa-2aff-4b27-9de7-69071750e4de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41b4d9513f204230a8ff27614eb1a642', 'request_id': 'req-81f031aa-2aff-4b27-9de7-69071750e4de'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:43:40.961558 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-81f031aa-2aff-4b27-9de7-69071750e4de admin admin] The token cache is being invalidate because project 41b4d9513f204230a8ff27614eb1a642 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:43:40.961847 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-81f031aa-2aff-4b27-9de7-69071750e4de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:43:40.962510 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-81f031aa-2aff-4b27-9de7-69071750e4de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:43:40.964103 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1323/2670] 10.222.0.102 () {58 vars in 1238 bytes} [Thu Sep 1 17:43:40 2022] DELETE /identity/v3/projects/41b4d9513f204230a8ff27614eb1a642 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:43:50.619220 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=29,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:43:50.642501 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99b7c95-c167-40c3-b378-902c62ef47ae admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:43:50.643844 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-b99b7c95-c167-40c3-b378-902c62ef47ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:43:50.701424 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99b7c95-c167-40c3-b378-902c62ef47ae admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:43:50.750352 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46934ca-a269-4f3a-b332-7aa5a9159fb2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:43:50.751021 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46934ca-a269-4f3a-b332-7aa5a9159fb2 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:43:50.751303 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46934ca-a269-4f3a-b332-7aa5a9159fb2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:43:50.751580 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46934ca-a269-4f3a-b332-7aa5a9159fb2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:43:50.753584 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-f46934ca-a269-4f3a-b332-7aa5a9159fb2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:43:50.823115 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46934ca-a269-4f3a-b332-7aa5a9159fb2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:43:50.824584 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46934ca-a269-4f3a-b332-7aa5a9159fb2 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:43:50.836750 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1348/2671] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:43:50 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 17:43:50.958444 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=51,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:43:53.940979 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f031aa-2aff-4b27-9de7-69071750e4de admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:43:53.942555 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-81f031aa-2aff-4b27-9de7-69071750e4de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:43:53.956222 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f031aa-2aff-4b27-9de7-69071750e4de admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:43:53.960748 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebb6f85-057f-4487-bcca-bf83bffd11b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:43:53.961404 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebb6f85-057f-4487-bcca-bf83bffd11b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:43:53.961679 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebb6f85-057f-4487-bcca-bf83bffd11b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:43:53.961947 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebb6f85-057f-4487-bcca-bf83bffd11b6 admin admin] PATH_INFO: `/v3/users/ae0aab3e503249d49ff9473b4990ff63` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:43:53.963056 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebb6f85-057f-4487-bcca-bf83bffd11b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae0aab3e503249d49ff9473b4990ff63)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:43:53.964953 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebb6f85-057f-4487-bcca-bf83bffd11b6 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:43:54.011504 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-8ebb6f85-057f-4487-bcca-bf83bffd11b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae0aab3e503249d49ff9473b4990ff63'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:43:54.012075 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-8ebb6f85-057f-4487-bcca-bf83bffd11b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae0aab3e503249d49ff9473b4990ff63'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:43:54.024043 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-8ebb6f85-057f-4487-bcca-bf83bffd11b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae0aab3e503249d49ff9473b4990ff63'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:43:54.031422 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-8ebb6f85-057f-4487-bcca-bf83bffd11b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae0aab3e503249d49ff9473b4990ff63'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:43:54.032154 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-8ebb6f85-057f-4487-bcca-bf83bffd11b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae0aab3e503249d49ff9473b4990ff63'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:43:54.044695 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1324/2672] 10.222.0.102 () {58 vars in 1229 bytes} [Thu Sep 1 17:43:53 2022] DELETE /identity/v3/users/ae0aab3e503249d49ff9473b4990ff63 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:43:54.226918 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46934ca-a269-4f3a-b332-7aa5a9159fb2 service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:43:54.236899 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46934ca-a269-4f3a-b332-7aa5a9159fb2 service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:43:54.295354 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343eab37-278f-42c2-8487-18329a9c8e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:43:54.295913 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343eab37-278f-42c2-8487-18329a9c8e3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:43:54.296192 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343eab37-278f-42c2-8487-18329a9c8e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:43:54.296461 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343eab37-278f-42c2-8487-18329a9c8e3d admin admin] PATH_INFO: `/v3/projects/19bb1cbe315441258828677997dd6fdf` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:43:54.297601 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343eab37-278f-42c2-8487-18329a9c8e3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=19bb1cbe315441258828677997dd6fdf)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:43:54.299490 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343eab37-278f-42c2-8487-18329a9c8e3d admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:43:54.307149 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:43:54.345845 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-343eab37-278f-42c2-8487-18329a9c8e3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19bb1cbe315441258828677997dd6fdf', 'request_id': 'req-343eab37-278f-42c2-8487-18329a9c8e3d'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:43:54.353508 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-343eab37-278f-42c2-8487-18329a9c8e3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19bb1cbe315441258828677997dd6fdf', 'request_id': 'req-343eab37-278f-42c2-8487-18329a9c8e3d'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:43:54.359744 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-343eab37-278f-42c2-8487-18329a9c8e3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19bb1cbe315441258828677997dd6fdf', 'request_id': 'req-343eab37-278f-42c2-8487-18329a9c8e3d'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:43:54.369978 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-343eab37-278f-42c2-8487-18329a9c8e3d admin admin] The token cache is being invalidate because project 19bb1cbe315441258828677997dd6fdf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:43:54.370307 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-343eab37-278f-42c2-8487-18329a9c8e3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:43:54.370977 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-343eab37-278f-42c2-8487-18329a9c8e3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:43:54.373402 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1349/2673] 10.222.0.102 () {58 vars in 1238 bytes} [Thu Sep 1 17:43:54 2022] DELETE /identity/v3/projects/19bb1cbe315441258828677997dd6fdf => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:43:54.378234 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6d6c82-aa34-4e2a-a57e-6bfe8bcb8c9e None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:43:54.378522 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6d6c82-aa34-4e2a-a57e-6bfe8bcb8c9e None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:43:54.378803 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6d6c82-aa34-4e2a-a57e-6bfe8bcb8c9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:43:54.398627 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-4a6d6c82-aa34-4e2a-a57e-6bfe8bcb8c9e None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:43:54.399827 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-4a6d6c82-aa34-4e2a-a57e-6bfe8bcb8c9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:43:54.456986 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1325/2674] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:43:54 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:43:54.464882 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a141743c-304b-4f88-aaf7-7c83f0180d4a None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:43:54.465170 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a141743c-304b-4f88-aaf7-7c83f0180d4a None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:43:54.465482 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a141743c-304b-4f88-aaf7-7c83f0180d4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:43:54.486280 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-a141743c-304b-4f88-aaf7-7c83f0180d4a None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:43:54.487470 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-a141743c-304b-4f88-aaf7-7c83f0180d4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:43:54.495106 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1350/2675] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:43:54 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:43:54.499269 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6d6c82-aa34-4e2a-a57e-6bfe8bcb8c9e None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:43:54.508866 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6d6c82-aa34-4e2a-a57e-6bfe8bcb8c9e None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:43:54.513962 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6375c7ce-ded8-45cf-ac11-917f7180a02d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:43:54.514551 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6375c7ce-ded8-45cf-ac11-917f7180a02d admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:43:54.514829 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6375c7ce-ded8-45cf-ac11-917f7180a02d admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:43:54.515102 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6375c7ce-ded8-45cf-ac11-917f7180a02d admin admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:43:54.515745 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6375c7ce-ded8-45cf-ac11-917f7180a02d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:43:54.516052 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6375c7ce-ded8-45cf-ac11-917f7180a02d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:43:54.517684 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6375c7ce-ded8-45cf-ac11-917f7180a02d admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:43:54.530064 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1326/2676] 10.222.0.102 () {58 vars in 1158 bytes} [Thu Sep 1 17:43:54 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:43:54.534180 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a141743c-304b-4f88-aaf7-7c83f0180d4a None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:43:54.543206 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a141743c-304b-4f88-aaf7-7c83f0180d4a None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:43:54.547904 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a5c91e-f30e-424e-8429-8eeef86f4172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:43:54.548485 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a5c91e-f30e-424e-8429-8eeef86f4172 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:43:54.548763 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a5c91e-f30e-424e-8429-8eeef86f4172 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:43:54.549030 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a5c91e-f30e-424e-8429-8eeef86f4172 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:43:54.549705 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a5c91e-f30e-424e-8429-8eeef86f4172 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:43:54.551568 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a5c91e-f30e-424e-8429-8eeef86f4172 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:43:54.586605 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1351/2677] 10.222.0.102 () {60 vars in 1158 bytes} [Thu Sep 1 17:43:54 2022] POST /identity/v3/projects => generated 374 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:43:54.591769 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6375c7ce-ded8-45cf-ac11-917f7180a02d admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:43:54.601994 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6375c7ce-ded8-45cf-ac11-917f7180a02d admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:43:54.657913 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633da816-e3c6-4974-ac85-820797cfd0c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:43:54.658520 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633da816-e3c6-4974-ac85-820797cfd0c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:43:54.658798 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633da816-e3c6-4974-ac85-820797cfd0c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:43:54.659173 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633da816-e3c6-4974-ac85-820797cfd0c6 admin admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:43:54.659718 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633da816-e3c6-4974-ac85-820797cfd0c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:43:54.661684 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633da816-e3c6-4974-ac85-820797cfd0c6 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:43:54.669902 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:43:54.689151 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1327/2678] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:43:54 2022] POST /identity/v3/users => generated 288 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:43:54.690322 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a5c91e-f30e-424e-8429-8eeef86f4172 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:43:54.702643 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a5c91e-f30e-424e-8429-8eeef86f4172 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:43:54.707203 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5ccf68-c00c-419e-9572-58c468f13c1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:43:54.707831 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5ccf68-c00c-419e-9572-58c468f13c1e admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:43:54.708109 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5ccf68-c00c-419e-9572-58c468f13c1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:43:54.708380 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5ccf68-c00c-419e-9572-58c468f13c1e admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:43:54.709013 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5ccf68-c00c-419e-9572-58c468f13c1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:43:54.710623 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5ccf68-c00c-419e-9572-58c468f13c1e admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:43:54.721498 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1352/2679] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:43:54 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 17:43:54.725591 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633da816-e3c6-4974-ac85-820797cfd0c6 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:43:54.735348 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633da816-e3c6-4974-ac85-820797cfd0c6 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:43:54.739917 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837ee57e-8c5d-486f-96cc-18d4e911c2df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:43:54.740500 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837ee57e-8c5d-486f-96cc-18d4e911c2df admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:43:54.740775 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837ee57e-8c5d-486f-96cc-18d4e911c2df admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:43:54.741065 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837ee57e-8c5d-486f-96cc-18d4e911c2df admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:43:54.741698 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837ee57e-8c5d-486f-96cc-18d4e911c2df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:43:54.743456 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837ee57e-8c5d-486f-96cc-18d4e911c2df admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:43:54.754019 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1328/2680] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:43:54 2022] GET /identity/v3/roles => generated 1427 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:43:54.758177 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5ccf68-c00c-419e-9572-58c468f13c1e admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:43:54.769785 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5ccf68-c00c-419e-9572-58c468f13c1e admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:43:54.774694 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2636e359-f54f-40e7-ba91-072f70530fa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:43:54.775353 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2636e359-f54f-40e7-ba91-072f70530fa5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:43:54.775638 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2636e359-f54f-40e7-ba91-072f70530fa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:43:54.775914 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2636e359-f54f-40e7-ba91-072f70530fa5 admin admin] PATH_INFO: `/v3/projects/9078902a80fe43598d2ef993c60fe908/users/e75eed4c0f6c4b83b7bc8daae0898764/roles/b75cb360925e4ba5a65f14965c103ffd` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:43:54.790916 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2636e359-f54f-40e7-ba91-072f70530fa5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9078902a80fe43598d2ef993c60fe908, user_id=e75eed4c0f6c4b83b7bc8daae0898764, role_id=b75cb360925e4ba5a65f14965c103ffd)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:43:54.793613 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2636e359-f54f-40e7-ba91-072f70530fa5 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:43:54.806323 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1353/2681] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:43:54 2022] PUT /identity/v3/projects/9078902a80fe43598d2ef993c60fe908/users/e75eed4c0f6c4b83b7bc8daae0898764/roles/b75cb360925e4ba5a65f14965c103ffd => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:43:54.818128 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f76aea8-ace8-4017-b461-6ac5ab62bf68 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:43:54.818418 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f76aea8-ace8-4017-b461-6ac5ab62bf68 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:43:54.818693 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f76aea8-ace8-4017-b461-6ac5ab62bf68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:43:54.847632 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-1f76aea8-ace8-4017-b461-6ac5ab62bf68 None None] MFA Rules not processed for user `e75eed4c0f6c4b83b7bc8daae0898764`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:43:54.848858 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-1f76aea8-ace8-4017-b461-6ac5ab62bf68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:43:54.916052 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1329/2682] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:43:54 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:43:54.927751 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2636e359-f54f-40e7-ba91-072f70530fa5 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:43:54.929604 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-2636e359-f54f-40e7-ba91-072f70530fa5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:43:55.010776 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2636e359-f54f-40e7-ba91-072f70530fa5 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:43:55.017506 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9345d33e-b787-4ae4-8a1e-629c4d784f59 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f8293316baf4774b62fc11d55d50865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:43:55.018136 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9345d33e-b787-4ae4-8a1e-629c4d784f59 service nova] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:43:55.018444 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9345d33e-b787-4ae4-8a1e-629c4d784f59 service nova] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:43:55.018722 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9345d33e-b787-4ae4-8a1e-629c4d784f59 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:43:55.027576 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9345d33e-b787-4ae4-8a1e-629c4d784f59 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:43:55.029238 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9345d33e-b787-4ae4-8a1e-629c4d784f59 service nova] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:43:55.038175 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1354/2683] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:43:54 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:43:57.712905 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f76aea8-ace8-4017-b461-6ac5ab62bf68 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:43:57.772754 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f76aea8-ace8-4017-b461-6ac5ab62bf68 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:43:57.777198 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad471f25-a06d-4add-98b6-e879fd1dea0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:43:57.777835 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad471f25-a06d-4add-98b6-e879fd1dea0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:43:57.778115 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad471f25-a06d-4add-98b6-e879fd1dea0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:43:57.778385 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad471f25-a06d-4add-98b6-e879fd1dea0b admin admin] PATH_INFO: `/v3/users/e75eed4c0f6c4b83b7bc8daae0898764` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:43:57.780023 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad471f25-a06d-4add-98b6-e879fd1dea0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=e75eed4c0f6c4b83b7bc8daae0898764)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:43:57.781947 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad471f25-a06d-4add-98b6-e879fd1dea0b admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:43:57.827861 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-ad471f25-a06d-4add-98b6-e879fd1dea0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e75eed4c0f6c4b83b7bc8daae0898764'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:43:57.828537 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-ad471f25-a06d-4add-98b6-e879fd1dea0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e75eed4c0f6c4b83b7bc8daae0898764'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:43:57.840579 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-ad471f25-a06d-4add-98b6-e879fd1dea0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e75eed4c0f6c4b83b7bc8daae0898764'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:43:57.845029 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:INSERT=4,keystone:SELECT=86,keystone:DELETE=10 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:43:57.849923 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-ad471f25-a06d-4add-98b6-e879fd1dea0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e75eed4c0f6c4b83b7bc8daae0898764'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:43:57.850720 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-ad471f25-a06d-4add-98b6-e879fd1dea0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e75eed4c0f6c4b83b7bc8daae0898764'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:43:57.865033 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1330/2684] 10.222.0.102 () {58 vars in 1229 bytes} [Thu Sep 1 17:43:57 2022] DELETE /identity/v3/users/e75eed4c0f6c4b83b7bc8daae0898764 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:43:57.869525 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac83ecd-d362-4982-99a5-1a20276718d7 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:43:57.869980 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac83ecd-d362-4982-99a5-1a20276718d7 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:43:57.870149 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac83ecd-d362-4982-99a5-1a20276718d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:43:57.889886 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-fac83ecd-d362-4982-99a5-1a20276718d7 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:43:57.891107 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-fac83ecd-d362-4982-99a5-1a20276718d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:43:57.894908 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:43:57.957196 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1355/2685] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:43:57 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:43:57.970906 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad471f25-a06d-4add-98b6-e879fd1dea0b admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:43:57.972526 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-ad471f25-a06d-4add-98b6-e879fd1dea0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:43:58.033919 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad471f25-a06d-4add-98b6-e879fd1dea0b admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:43:58.037685 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98857266-4b2e-42f0-8032-619b276c8477 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:43:58.038287 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98857266-4b2e-42f0-8032-619b276c8477 service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:43:58.038568 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98857266-4b2e-42f0-8032-619b276c8477 service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:43:58.038841 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98857266-4b2e-42f0-8032-619b276c8477 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:43:58.047749 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98857266-4b2e-42f0-8032-619b276c8477 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:43:58.049169 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98857266-4b2e-42f0-8032-619b276c8477 service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:43:58.058540 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1331/2686] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:43:57 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 17:43:58.336522 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac83ecd-d362-4982-99a5-1a20276718d7 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:43:58.338155 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-fac83ecd-d362-4982-99a5-1a20276718d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:43:58.351520 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac83ecd-d362-4982-99a5-1a20276718d7 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:43:58.356323 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fca8cd6-a46e-43ec-b41b-9d6fbf060da1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:43:58.356954 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fca8cd6-a46e-43ec-b41b-9d6fbf060da1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:43:58.357275 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fca8cd6-a46e-43ec-b41b-9d6fbf060da1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:43:58.357542 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fca8cd6-a46e-43ec-b41b-9d6fbf060da1 admin admin] PATH_INFO: `/v3/projects/9078902a80fe43598d2ef993c60fe908` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:43:58.358670 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fca8cd6-a46e-43ec-b41b-9d6fbf060da1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9078902a80fe43598d2ef993c60fe908)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:43:58.360583 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fca8cd6-a46e-43ec-b41b-9d6fbf060da1 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:43:58.411701 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9fca8cd6-a46e-43ec-b41b-9d6fbf060da1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9078902a80fe43598d2ef993c60fe908', 'request_id': 'req-9fca8cd6-a46e-43ec-b41b-9d6fbf060da1'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:43:58.419202 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9fca8cd6-a46e-43ec-b41b-9d6fbf060da1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9078902a80fe43598d2ef993c60fe908', 'request_id': 'req-9fca8cd6-a46e-43ec-b41b-9d6fbf060da1'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:43:58.425725 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9fca8cd6-a46e-43ec-b41b-9d6fbf060da1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9078902a80fe43598d2ef993c60fe908', 'request_id': 'req-9fca8cd6-a46e-43ec-b41b-9d6fbf060da1'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:43:58.436251 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9fca8cd6-a46e-43ec-b41b-9d6fbf060da1 admin admin] The token cache is being invalidate because project 9078902a80fe43598d2ef993c60fe908 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:43:58.436545 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9fca8cd6-a46e-43ec-b41b-9d6fbf060da1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:43:58.437204 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9fca8cd6-a46e-43ec-b41b-9d6fbf060da1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:43:58.438716 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1356/2687] 10.222.0.102 () {58 vars in 1238 bytes} [Thu Sep 1 17:43:58 2022] DELETE /identity/v3/projects/9078902a80fe43598d2ef993c60fe908 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:43:58.443577 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d2e305-eb30-4055-8cb6-9aae13029cd2 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:43:58.443868 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d2e305-eb30-4055-8cb6-9aae13029cd2 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:43:58.444163 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d2e305-eb30-4055-8cb6-9aae13029cd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:43:58.464121 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-61d2e305-eb30-4055-8cb6-9aae13029cd2 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:43:58.465260 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-61d2e305-eb30-4055-8cb6-9aae13029cd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:43:58.529979 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1332/2688] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:43:58 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:43:58.538246 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e336db-4f2e-4c83-aff0-c1c185bea3b3 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:43:58.538546 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e336db-4f2e-4c83-aff0-c1c185bea3b3 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:43:58.538826 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e336db-4f2e-4c83-aff0-c1c185bea3b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:43:58.562159 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-30e336db-4f2e-4c83-aff0-c1c185bea3b3 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:43:58.563474 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-30e336db-4f2e-4c83-aff0-c1c185bea3b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:43:58.576938 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1357/2689] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:43:58 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:43:58.579328 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d2e305-eb30-4055-8cb6-9aae13029cd2 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:43:58.590759 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d2e305-eb30-4055-8cb6-9aae13029cd2 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:43:58.597014 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329d3d23-0c68-4e90-a128-78b6c21f19f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:43:58.597684 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329d3d23-0c68-4e90-a128-78b6c21f19f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:43:58.597995 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329d3d23-0c68-4e90-a128-78b6c21f19f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:43:58.598277 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329d3d23-0c68-4e90-a128-78b6c21f19f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:43:58.599116 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329d3d23-0c68-4e90-a128-78b6c21f19f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:43:58.599442 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329d3d23-0c68-4e90-a128-78b6c21f19f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:43:58.601119 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329d3d23-0c68-4e90-a128-78b6c21f19f9 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:43:58.614552 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1333/2690] 10.222.0.102 () {58 vars in 1158 bytes} [Thu Sep 1 17:43:58 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:43:58.619077 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e336db-4f2e-4c83-aff0-c1c185bea3b3 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:43:58.630439 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e336db-4f2e-4c83-aff0-c1c185bea3b3 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:43:58.635308 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea5a788-e404-46ac-85b8-16b03a3f01ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:43:58.635950 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea5a788-e404-46ac-85b8-16b03a3f01ed admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:43:58.636231 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea5a788-e404-46ac-85b8-16b03a3f01ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:43:58.636505 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea5a788-e404-46ac-85b8-16b03a3f01ed admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:43:58.637150 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea5a788-e404-46ac-85b8-16b03a3f01ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:43:58.639653 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea5a788-e404-46ac-85b8-16b03a3f01ed admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:43:58.670312 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1358/2691] 10.222.0.102 () {60 vars in 1158 bytes} [Thu Sep 1 17:43:58 2022] POST /identity/v3/projects => generated 374 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:43:58.674511 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329d3d23-0c68-4e90-a128-78b6c21f19f9 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:43:58.686909 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329d3d23-0c68-4e90-a128-78b6c21f19f9 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:43:58.749507 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b90ee63-bee8-474a-8ea8-d5a05fb8aac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:43:58.750149 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b90ee63-bee8-474a-8ea8-d5a05fb8aac0 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:43:58.750531 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b90ee63-bee8-474a-8ea8-d5a05fb8aac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:43:58.750809 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b90ee63-bee8-474a-8ea8-d5a05fb8aac0 admin admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:43:58.751433 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b90ee63-bee8-474a-8ea8-d5a05fb8aac0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:43:58.753350 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b90ee63-bee8-474a-8ea8-d5a05fb8aac0 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:43:58.775516 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1334/2692] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:43:58 2022] POST /identity/v3/users => generated 288 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:43:58.779767 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea5a788-e404-46ac-85b8-16b03a3f01ed admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:43:58.790912 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea5a788-e404-46ac-85b8-16b03a3f01ed admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:43:58.796594 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12602f2b-5eae-4fa4-aec1-4d4fa1b9342f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:43:58.797208 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12602f2b-5eae-4fa4-aec1-4d4fa1b9342f admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:43:58.797557 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12602f2b-5eae-4fa4-aec1-4d4fa1b9342f admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:43:58.797834 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12602f2b-5eae-4fa4-aec1-4d4fa1b9342f admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:43:58.798482 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12602f2b-5eae-4fa4-aec1-4d4fa1b9342f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:43:58.800101 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12602f2b-5eae-4fa4-aec1-4d4fa1b9342f admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:43:58.811750 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1359/2693] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:43:58 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:43:58.815650 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b90ee63-bee8-474a-8ea8-d5a05fb8aac0 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:43:58.825731 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b90ee63-bee8-474a-8ea8-d5a05fb8aac0 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:43:58.831100 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79c8181-a51e-4ab5-8e76-6df026b93f50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:43:58.831698 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79c8181-a51e-4ab5-8e76-6df026b93f50 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:43:58.831983 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79c8181-a51e-4ab5-8e76-6df026b93f50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:43:58.832254 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79c8181-a51e-4ab5-8e76-6df026b93f50 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:43:58.832888 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79c8181-a51e-4ab5-8e76-6df026b93f50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:43:58.834502 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79c8181-a51e-4ab5-8e76-6df026b93f50 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:43:58.835859 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:43:58.846928 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1335/2694] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:43:58 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 17:43:58.851110 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12602f2b-5eae-4fa4-aec1-4d4fa1b9342f admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:43:58.861518 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12602f2b-5eae-4fa4-aec1-4d4fa1b9342f admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:43:58.866117 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac182e9-f5d5-4fdd-a77a-07c9c58ddfbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:43:58.866798 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac182e9-f5d5-4fdd-a77a-07c9c58ddfbf admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:43:58.867076 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac182e9-f5d5-4fdd-a77a-07c9c58ddfbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:43:58.867541 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac182e9-f5d5-4fdd-a77a-07c9c58ddfbf admin admin] PATH_INFO: `/v3/projects/ed870e25f9d545e8b027fe724e6eb285/users/1755e51704664616b77422cc0d49b09f/roles/b75cb360925e4ba5a65f14965c103ffd` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:43:58.883445 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac182e9-f5d5-4fdd-a77a-07c9c58ddfbf admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed870e25f9d545e8b027fe724e6eb285, user_id=1755e51704664616b77422cc0d49b09f, role_id=b75cb360925e4ba5a65f14965c103ffd)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:43:58.886227 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac182e9-f5d5-4fdd-a77a-07c9c58ddfbf admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:43:58.900318 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1360/2695] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:43:58 2022] PUT /identity/v3/projects/ed870e25f9d545e8b027fe724e6eb285/users/1755e51704664616b77422cc0d49b09f/roles/b75cb360925e4ba5a65f14965c103ffd => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:43:58.910376 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c9ee58-ae6b-4df6-bc4b-ceb6bbf73c9d None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:43:58.910661 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c9ee58-ae6b-4df6-bc4b-ceb6bbf73c9d None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:43:58.910957 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c9ee58-ae6b-4df6-bc4b-ceb6bbf73c9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:43:58.939065 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-45c9ee58-ae6b-4df6-bc4b-ceb6bbf73c9d None None] MFA Rules not processed for user `1755e51704664616b77422cc0d49b09f`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:43:58.940328 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-45c9ee58-ae6b-4df6-bc4b-ceb6bbf73c9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:43:59.012256 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1336/2696] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:43:58 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:43:59.016459 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac182e9-f5d5-4fdd-a77a-07c9c58ddfbf admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:43:59.048968 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:43:59.087168 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac182e9-f5d5-4fdd-a77a-07c9c58ddfbf admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:43:59.092078 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8570e48c-00e2-4c4c-8f3f-2aa36169c8dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:43:59.092780 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8570e48c-00e2-4c4c-8f3f-2aa36169c8dc admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:43:59.093134 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8570e48c-00e2-4c4c-8f3f-2aa36169c8dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:43:59.093460 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8570e48c-00e2-4c4c-8f3f-2aa36169c8dc admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:43:59.094109 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8570e48c-00e2-4c4c-8f3f-2aa36169c8dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:43:59.096281 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8570e48c-00e2-4c4c-8f3f-2aa36169c8dc admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:43:59.124421 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1361/2697] 10.222.0.102 () {60 vars in 1158 bytes} [Thu Sep 1 17:43:59 2022] POST /identity/v3/projects => generated 374 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:43:59.128553 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c9ee58-ae6b-4df6-bc4b-ceb6bbf73c9d None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:43:59.139727 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c9ee58-ae6b-4df6-bc4b-ceb6bbf73c9d None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:43:59.198705 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58311432-7135-4dfe-be64-c3954a515e61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:43:59.199313 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58311432-7135-4dfe-be64-c3954a515e61 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:43:59.199592 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58311432-7135-4dfe-be64-c3954a515e61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:43:59.199863 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58311432-7135-4dfe-be64-c3954a515e61 admin admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:43:59.200497 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58311432-7135-4dfe-be64-c3954a515e61 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:43:59.203097 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58311432-7135-4dfe-be64-c3954a515e61 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:43:59.237982 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1337/2698] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:43:59 2022] POST /identity/v3/users => generated 294 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:43:59.239132 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8570e48c-00e2-4c4c-8f3f-2aa36169c8dc admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:43:59.251378 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8570e48c-00e2-4c4c-8f3f-2aa36169c8dc admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:43:59.256726 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947e8ddb-b3c5-40e8-8168-7e7e1246a9be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:43:59.257431 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947e8ddb-b3c5-40e8-8168-7e7e1246a9be admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:43:59.257710 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947e8ddb-b3c5-40e8-8168-7e7e1246a9be admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:43:59.258006 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947e8ddb-b3c5-40e8-8168-7e7e1246a9be admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:43:59.259225 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947e8ddb-b3c5-40e8-8168-7e7e1246a9be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:43:59.261031 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947e8ddb-b3c5-40e8-8168-7e7e1246a9be admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:43:59.284449 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1362/2699] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:43:59 2022] GET /identity/v3/roles => generated 1427 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:43:59.285739 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58311432-7135-4dfe-be64-c3954a515e61 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:43:59.297469 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58311432-7135-4dfe-be64-c3954a515e61 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:43:59.306002 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ffdd2b-dfa9-4f1a-9dbb-cb84fd99cea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:43:59.306561 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ffdd2b-dfa9-4f1a-9dbb-cb84fd99cea9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:43:59.306929 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ffdd2b-dfa9-4f1a-9dbb-cb84fd99cea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:43:59.307105 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ffdd2b-dfa9-4f1a-9dbb-cb84fd99cea9 admin admin] PATH_INFO: `/v3/projects/c26adac2e00949cfa1c27e475a12e79d/users/c8ac3f780ca34123a1372bea4a4cebcb/roles/36577a6f46d642d4b81805ede9872490` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:43:59.329247 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ffdd2b-dfa9-4f1a-9dbb-cb84fd99cea9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c26adac2e00949cfa1c27e475a12e79d, user_id=c8ac3f780ca34123a1372bea4a4cebcb, role_id=36577a6f46d642d4b81805ede9872490)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:43:59.332352 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ffdd2b-dfa9-4f1a-9dbb-cb84fd99cea9 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:43:59.352907 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1338/2700] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:43:59 2022] PUT /identity/v3/projects/c26adac2e00949cfa1c27e475a12e79d/users/c8ac3f780ca34123a1372bea4a4cebcb/roles/36577a6f46d642d4b81805ede9872490 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:43:59.354072 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947e8ddb-b3c5-40e8-8168-7e7e1246a9be admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:43:59.366096 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947e8ddb-b3c5-40e8-8168-7e7e1246a9be admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:43:59.441283 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd852732-c399-497d-85a3-85a5b1a5dc3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:43:59.441938 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd852732-c399-497d-85a3-85a5b1a5dc3b admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:43:59.442741 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd852732-c399-497d-85a3-85a5b1a5dc3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:43:59.442741 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd852732-c399-497d-85a3-85a5b1a5dc3b admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:43:59.443175 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd852732-c399-497d-85a3-85a5b1a5dc3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:43:59.444790 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd852732-c399-497d-85a3-85a5b1a5dc3b admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:43:59.457900 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1363/2701] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:43:59 2022] GET /identity/v3/roles => generated 1427 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 17:43:59.462413 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ffdd2b-dfa9-4f1a-9dbb-cb84fd99cea9 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:43:59.477862 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ffdd2b-dfa9-4f1a-9dbb-cb84fd99cea9 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:43:59.483642 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4016492d-1746-42ec-9f89-8b26aab9f2a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:43:59.484451 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4016492d-1746-42ec-9f89-8b26aab9f2a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:43:59.484918 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4016492d-1746-42ec-9f89-8b26aab9f2a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:43:59.485374 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4016492d-1746-42ec-9f89-8b26aab9f2a3 admin admin] PATH_INFO: `/v3/projects/c26adac2e00949cfa1c27e475a12e79d/users/c8ac3f780ca34123a1372bea4a4cebcb/roles/36577a6f46d642d4b81805ede9872490` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:43:59.489338 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4016492d-1746-42ec-9f89-8b26aab9f2a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c26adac2e00949cfa1c27e475a12e79d, user_id=c8ac3f780ca34123a1372bea4a4cebcb, role_id=36577a6f46d642d4b81805ede9872490)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:43:59.493289 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4016492d-1746-42ec-9f89-8b26aab9f2a3 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:43:59.510362 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1339/2702] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:43:59 2022] PUT /identity/v3/projects/c26adac2e00949cfa1c27e475a12e79d/users/c8ac3f780ca34123a1372bea4a4cebcb/roles/36577a6f46d642d4b81805ede9872490 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:43:59.522630 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fde1bb-7633-4b1a-95a2-bab86cf534a9 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:43:59.523062 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fde1bb-7633-4b1a-95a2-bab86cf534a9 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:43:59.523350 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fde1bb-7633-4b1a-95a2-bab86cf534a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:43:59.552793 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-29fde1bb-7633-4b1a-95a2-bab86cf534a9 None None] MFA Rules not processed for user `c8ac3f780ca34123a1372bea4a4cebcb`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:43:59.554668 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-29fde1bb-7633-4b1a-95a2-bab86cf534a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:43:59.626286 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1364/2703] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:43:59 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:43:59.637634 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4016492d-1746-42ec-9f89-8b26aab9f2a3 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:43:59.639469 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-4016492d-1746-42ec-9f89-8b26aab9f2a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:43:59.685489 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:43:59.702775 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4016492d-1746-42ec-9f89-8b26aab9f2a3 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:43:59.706526 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14343df0-4610-4f9d-a4e1-598ac813773c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '523ed589872a45a09b6a7675a7fb0b75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:43:59.707139 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14343df0-4610-4f9d-a4e1-598ac813773c service cinder] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:43:59.707411 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14343df0-4610-4f9d-a4e1-598ac813773c service cinder] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:43:59.707683 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14343df0-4610-4f9d-a4e1-598ac813773c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:43:59.717782 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14343df0-4610-4f9d-a4e1-598ac813773c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:43:59.719216 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14343df0-4610-4f9d-a4e1-598ac813773c service cinder] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:43:59.730088 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1340/2704] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:43:59 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:43:59.824873 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fde1bb-7633-4b1a-95a2-bab86cf534a9 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:43:59.836116 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fde1bb-7633-4b1a-95a2-bab86cf534a9 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:43:59.860682 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4607ef8c-4363-47bf-ae35-ca4ee27e3c5a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '523ed589872a45a09b6a7675a7fb0b75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:43:59.861505 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4607ef8c-4363-47bf-ae35-ca4ee27e3c5a service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:43:59.862179 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4607ef8c-4363-47bf-ae35-ca4ee27e3c5a service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:43:59.862179 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4607ef8c-4363-47bf-ae35-ca4ee27e3c5a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:43:59.889406 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:43:59.942919 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4607ef8c-4363-47bf-ae35-ca4ee27e3c5a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:43:59.944710 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4607ef8c-4363-47bf-ae35-ca4ee27e3c5a service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:43:59.954809 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1365/2705] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:43:59 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:44:09.300353 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14343df0-4610-4f9d-a4e1-598ac813773c service cinder] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:09.301812 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-14343df0-4610-4f9d-a4e1-598ac813773c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:09.368301 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14343df0-4610-4f9d-a4e1-598ac813773c service cinder] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:09.372081 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87bcc04-a449-4e2e-abad-0391af1517da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:09.372685 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87bcc04-a449-4e2e-abad-0391af1517da service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:09.372964 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87bcc04-a449-4e2e-abad-0391af1517da service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:09.373287 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87bcc04-a449-4e2e-abad-0391af1517da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:09.375165 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-b87bcc04-a449-4e2e-abad-0391af1517da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:09.441474 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87bcc04-a449-4e2e-abad-0391af1517da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:09.443097 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87bcc04-a449-4e2e-abad-0391af1517da service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:09.453256 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1341/2706] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:44:09 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:44:09.949652 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=17 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:12.521064 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4607ef8c-4363-47bf-ae35-ca4ee27e3c5a service cinder] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:12.523334 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-4607ef8c-4363-47bf-ae35-ca4ee27e3c5a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:12.540371 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4607ef8c-4363-47bf-ae35-ca4ee27e3c5a service cinder] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:12.545736 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8f6e8b-9024-4bb9-90ab-515c28a37c72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:12.546615 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8f6e8b-9024-4bb9-90ab-515c28a37c72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:12.547276 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8f6e8b-9024-4bb9-90ab-515c28a37c72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:12.547814 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8f6e8b-9024-4bb9-90ab-515c28a37c72 admin admin] PATH_INFO: `/v3/users/26bf17f1fb8948008fed82f4ba33bf66` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:12.549373 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8f6e8b-9024-4bb9-90ab-515c28a37c72 admin admin] RBAC: Authorizing `identity:delete_user(user_id=26bf17f1fb8948008fed82f4ba33bf66)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:12.552297 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8f6e8b-9024-4bb9-90ab-515c28a37c72 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:12.620057 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-ef8f6e8b-9024-4bb9-90ab-515c28a37c72 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26bf17f1fb8948008fed82f4ba33bf66'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:12.633067 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-ef8f6e8b-9024-4bb9-90ab-515c28a37c72 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26bf17f1fb8948008fed82f4ba33bf66'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:12.641913 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-ef8f6e8b-9024-4bb9-90ab-515c28a37c72 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26bf17f1fb8948008fed82f4ba33bf66'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:12.642540 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-ef8f6e8b-9024-4bb9-90ab-515c28a37c72 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26bf17f1fb8948008fed82f4ba33bf66'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:12.643211 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-ef8f6e8b-9024-4bb9-90ab-515c28a37c72 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26bf17f1fb8948008fed82f4ba33bf66'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:12.658855 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1366/2707] 10.222.0.102 () {58 vars in 1229 bytes} [Thu Sep 1 17:44:12 2022] DELETE /identity/v3/users/26bf17f1fb8948008fed82f4ba33bf66 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:44:12.876257 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87bcc04-a449-4e2e-abad-0391af1517da service neutron] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:12.886614 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87bcc04-a449-4e2e-abad-0391af1517da service neutron] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:12.951194 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7d35fa-5f88-441f-a1b5-fe896ca87d53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:12.951744 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7d35fa-5f88-441f-a1b5-fe896ca87d53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:12.952020 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7d35fa-5f88-441f-a1b5-fe896ca87d53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:12.952929 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7d35fa-5f88-441f-a1b5-fe896ca87d53 admin admin] PATH_INFO: `/v3/projects/97796d8be4794e89a89965af17bb3fa9` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:12.954100 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7d35fa-5f88-441f-a1b5-fe896ca87d53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=97796d8be4794e89a89965af17bb3fa9)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:12.955978 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7d35fa-5f88-441f-a1b5-fe896ca87d53 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:12.991515 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:13.005591 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-4a7d35fa-5f88-441f-a1b5-fe896ca87d53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97796d8be4794e89a89965af17bb3fa9', 'request_id': 'req-4a7d35fa-5f88-441f-a1b5-fe896ca87d53'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:13.011313 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-4a7d35fa-5f88-441f-a1b5-fe896ca87d53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97796d8be4794e89a89965af17bb3fa9', 'request_id': 'req-4a7d35fa-5f88-441f-a1b5-fe896ca87d53'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:13.019608 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-4a7d35fa-5f88-441f-a1b5-fe896ca87d53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97796d8be4794e89a89965af17bb3fa9', 'request_id': 'req-4a7d35fa-5f88-441f-a1b5-fe896ca87d53'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:13.032098 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-4a7d35fa-5f88-441f-a1b5-fe896ca87d53 admin admin] The token cache is being invalidate because project 97796d8be4794e89a89965af17bb3fa9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:44:13.032412 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-4a7d35fa-5f88-441f-a1b5-fe896ca87d53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:13.033148 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-4a7d35fa-5f88-441f-a1b5-fe896ca87d53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:13.034784 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1342/2708] 10.222.0.102 () {58 vars in 1238 bytes} [Thu Sep 1 17:44:12 2022] DELETE /identity/v3/projects/97796d8be4794e89a89965af17bb3fa9 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:44:13.039610 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee1159c-ee07-46ee-b0ae-714bd0766417 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:13.039894 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee1159c-ee07-46ee-b0ae-714bd0766417 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:13.040166 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee1159c-ee07-46ee-b0ae-714bd0766417 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:13.059320 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-3ee1159c-ee07-46ee-b0ae-714bd0766417 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:13.060506 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-3ee1159c-ee07-46ee-b0ae-714bd0766417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:13.123765 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1367/2709] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:44:13 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:44:13.133336 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caea40b9-04c2-4f46-abbe-085c011af193 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:13.133642 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caea40b9-04c2-4f46-abbe-085c011af193 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:13.133920 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caea40b9-04c2-4f46-abbe-085c011af193 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:13.157750 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-caea40b9-04c2-4f46-abbe-085c011af193 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:13.158974 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-caea40b9-04c2-4f46-abbe-085c011af193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:13.167804 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1343/2710] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:44:13 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:44:13.176839 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee1159c-ee07-46ee-b0ae-714bd0766417 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:13.186586 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee1159c-ee07-46ee-b0ae-714bd0766417 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:13.191699 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd67eac-5037-4fce-b6e6-3debbec3d8d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:13.192286 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd67eac-5037-4fce-b6e6-3debbec3d8d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:13.192573 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd67eac-5037-4fce-b6e6-3debbec3d8d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:13.193166 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd67eac-5037-4fce-b6e6-3debbec3d8d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:13.193754 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd67eac-5037-4fce-b6e6-3debbec3d8d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:44:13.194073 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd67eac-5037-4fce-b6e6-3debbec3d8d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:13.195878 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd67eac-5037-4fce-b6e6-3debbec3d8d1 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:13.208962 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1368/2711] 10.222.0.102 () {58 vars in 1158 bytes} [Thu Sep 1 17:44:13 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:44:13.213806 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caea40b9-04c2-4f46-abbe-085c011af193 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:13.224363 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caea40b9-04c2-4f46-abbe-085c011af193 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:13.229543 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22fb073-3651-491e-b631-6d2269cda49d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:13.230374 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22fb073-3651-491e-b631-6d2269cda49d admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:13.230649 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22fb073-3651-491e-b631-6d2269cda49d admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:13.230920 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22fb073-3651-491e-b631-6d2269cda49d admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:13.231573 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22fb073-3651-491e-b631-6d2269cda49d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:13.233511 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22fb073-3651-491e-b631-6d2269cda49d admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:13.263518 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1344/2712] 10.222.0.102 () {60 vars in 1158 bytes} [Thu Sep 1 17:44:13 2022] POST /identity/v3/projects => generated 384 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:44:13.267489 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd67eac-5037-4fce-b6e6-3debbec3d8d1 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:13.277601 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd67eac-5037-4fce-b6e6-3debbec3d8d1 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:13.342650 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf42a68d-d755-4c2e-b627-d5fee8ebd86d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:13.343353 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf42a68d-d755-4c2e-b627-d5fee8ebd86d admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:13.343636 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf42a68d-d755-4c2e-b627-d5fee8ebd86d admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:13.343950 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf42a68d-d755-4c2e-b627-d5fee8ebd86d admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:13.344616 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf42a68d-d755-4c2e-b627-d5fee8ebd86d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:13.346880 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf42a68d-d755-4c2e-b627-d5fee8ebd86d admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:13.360598 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:13.374310 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1369/2713] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:44:13 2022] POST /identity/v3/users => generated 293 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:44:13.378209 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22fb073-3651-491e-b631-6d2269cda49d admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:13.390806 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22fb073-3651-491e-b631-6d2269cda49d admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:13.395671 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d1d1b9-f698-4558-80d4-de692056dbac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:13.396371 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d1d1b9-f698-4558-80d4-de692056dbac admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:13.396663 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d1d1b9-f698-4558-80d4-de692056dbac admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:13.396938 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d1d1b9-f698-4558-80d4-de692056dbac admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:13.397622 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d1d1b9-f698-4558-80d4-de692056dbac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:13.399242 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d1d1b9-f698-4558-80d4-de692056dbac admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:13.414190 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1345/2714] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:44:13 2022] GET /identity/v3/roles => generated 1427 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 17:44:13.415407 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf42a68d-d755-4c2e-b627-d5fee8ebd86d admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:13.425959 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf42a68d-d755-4c2e-b627-d5fee8ebd86d admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:13.430761 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12319ad-33c0-4b52-b605-717804371d25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:13.431384 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12319ad-33c0-4b52-b605-717804371d25 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:13.431671 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12319ad-33c0-4b52-b605-717804371d25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:13.431947 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12319ad-33c0-4b52-b605-717804371d25 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:13.432605 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12319ad-33c0-4b52-b605-717804371d25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:13.434260 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12319ad-33c0-4b52-b605-717804371d25 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:13.445821 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1370/2715] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:44:13 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:44:13.450287 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d1d1b9-f698-4558-80d4-de692056dbac admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:13.462001 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d1d1b9-f698-4558-80d4-de692056dbac admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:13.466779 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296c32fa-e13a-4753-b9ef-5d3e640fbd01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:13.467445 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296c32fa-e13a-4753-b9ef-5d3e640fbd01 admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:13.467728 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296c32fa-e13a-4753-b9ef-5d3e640fbd01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:13.468629 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296c32fa-e13a-4753-b9ef-5d3e640fbd01 admin admin] PATH_INFO: `/v3/projects/de5f40e77a914f919f5bb15d11181f65/users/e7660433575a4daaaed075a6c8934934/roles/b75cb360925e4ba5a65f14965c103ffd` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:13.485207 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296c32fa-e13a-4753-b9ef-5d3e640fbd01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de5f40e77a914f919f5bb15d11181f65, user_id=e7660433575a4daaaed075a6c8934934, role_id=b75cb360925e4ba5a65f14965c103ffd)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:13.495585 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296c32fa-e13a-4753-b9ef-5d3e640fbd01 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:13.518395 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1346/2716] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:44:13 2022] PUT /identity/v3/projects/de5f40e77a914f919f5bb15d11181f65/users/e7660433575a4daaaed075a6c8934934/roles/b75cb360925e4ba5a65f14965c103ffd => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:44:13.525689 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ed35e4-6d10-4a7c-a15e-710fdac4328d None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:13.526271 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ed35e4-6d10-4a7c-a15e-710fdac4328d None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:13.526271 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ed35e4-6d10-4a7c-a15e-710fdac4328d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:13.547312 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-b6ed35e4-6d10-4a7c-a15e-710fdac4328d None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:13.548905 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-b6ed35e4-6d10-4a7c-a15e-710fdac4328d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:13.621994 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1371/2717] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:44:13 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:44:13.629587 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296c32fa-e13a-4753-b9ef-5d3e640fbd01 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:13.631681 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-296c32fa-e13a-4753-b9ef-5d3e640fbd01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:13.699676 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296c32fa-e13a-4753-b9ef-5d3e640fbd01 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:13.703602 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704df39c-dc7c-4aee-88d2-a18df8a238f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:13.704214 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704df39c-dc7c-4aee-88d2-a18df8a238f4 service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:13.704480 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704df39c-dc7c-4aee-88d2-a18df8a238f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:13.705032 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704df39c-dc7c-4aee-88d2-a18df8a238f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:13.714984 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704df39c-dc7c-4aee-88d2-a18df8a238f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:13.716414 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704df39c-dc7c-4aee-88d2-a18df8a238f4 service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:13.728872 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1347/2718] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:44:13 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 17:44:19.198168 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4bff0d-f9d1-43e8-bad3-89d8e8f60666 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:19.198695 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4bff0d-f9d1-43e8-bad3-89d8e8f60666 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:19.198824 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4bff0d-f9d1-43e8-bad3-89d8e8f60666 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:19.230241 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-2b4bff0d-f9d1-43e8-bad3-89d8e8f60666 None None] MFA Rules not processed for user `e7660433575a4daaaed075a6c8934934`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:19.231523 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-2b4bff0d-f9d1-43e8-bad3-89d8e8f60666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:19.303270 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1372/2719] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:44:19 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:44:19.318146 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704df39c-dc7c-4aee-88d2-a18df8a238f4 service neutron] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:19.319603 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-704df39c-dc7c-4aee-88d2-a18df8a238f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:19.440623 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:19.459573 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704df39c-dc7c-4aee-88d2-a18df8a238f4 service neutron] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:19.469402 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372dea86-8932-4d22-b2ec-befc09c60ac7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f8293316baf4774b62fc11d55d50865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:19.470056 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372dea86-8932-4d22-b2ec-befc09c60ac7 service nova] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:19.470349 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372dea86-8932-4d22-b2ec-befc09c60ac7 service nova] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:19.470643 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372dea86-8932-4d22-b2ec-befc09c60ac7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:19.481218 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372dea86-8932-4d22-b2ec-befc09c60ac7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:19.483048 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372dea86-8932-4d22-b2ec-befc09c60ac7 service nova] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:19.497052 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1348/2720] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:44:19 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 17:44:20.377420 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4bff0d-f9d1-43e8-bad3-89d8e8f60666 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:20.387401 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4bff0d-f9d1-43e8-bad3-89d8e8f60666 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:20.392637 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a0a257-5493-4387-9ffa-ea6a2e5311fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:20.393242 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a0a257-5493-4387-9ffa-ea6a2e5311fa service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:20.393965 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a0a257-5493-4387-9ffa-ea6a2e5311fa service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:20.394239 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a0a257-5493-4387-9ffa-ea6a2e5311fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:20.396265 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-92a0a257-5493-4387-9ffa-ea6a2e5311fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:20.410776 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a0a257-5493-4387-9ffa-ea6a2e5311fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:20.412333 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a0a257-5493-4387-9ffa-ea6a2e5311fa service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:20.424084 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1373/2721] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:44:20 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:44:20.877431 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4424c81-0f4f-4c1a-8b03-4ecbb721e0c6 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:20.877914 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4424c81-0f4f-4c1a-8b03-4ecbb721e0c6 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:20.878122 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4424c81-0f4f-4c1a-8b03-4ecbb721e0c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:20.907095 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-c4424c81-0f4f-4c1a-8b03-4ecbb721e0c6 None None] MFA Rules not processed for user `e7660433575a4daaaed075a6c8934934`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:20.910198 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-c4424c81-0f4f-4c1a-8b03-4ecbb721e0c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:20.920224 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1349/2722] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:44:20 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:44:22.023030 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a0a257-5493-4387-9ffa-ea6a2e5311fa service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:22.024604 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-92a0a257-5493-4387-9ffa-ea6a2e5311fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:22.097637 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:22.116122 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a0a257-5493-4387-9ffa-ea6a2e5311fa service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:22.121230 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c05c9b-ec51-41b5-ad2c-c3441f45e7e8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '134ef6039cef468bae0e3f3be79b8073', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:22.121873 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c05c9b-ec51-41b5-ad2c-c3441f45e7e8 service placement] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:22.122152 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c05c9b-ec51-41b5-ad2c-c3441f45e7e8 service placement] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:22.122421 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c05c9b-ec51-41b5-ad2c-c3441f45e7e8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:22.124154 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-88c05c9b-ec51-41b5-ad2c-c3441f45e7e8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:22.136940 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c05c9b-ec51-41b5-ad2c-c3441f45e7e8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:22.138570 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c05c9b-ec51-41b5-ad2c-c3441f45e7e8 service placement] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:22.150197 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1374/2723] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:44:22 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:44:27.706495 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4424c81-0f4f-4c1a-8b03-4ecbb721e0c6 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:27.708071 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-c4424c81-0f4f-4c1a-8b03-4ecbb721e0c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:27.722144 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4424c81-0f4f-4c1a-8b03-4ecbb721e0c6 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:27.727202 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685902ff-4a6c-403c-b78e-bab2d20d1c75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:27.727837 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685902ff-4a6c-403c-b78e-bab2d20d1c75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:27.728117 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685902ff-4a6c-403c-b78e-bab2d20d1c75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:27.728547 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685902ff-4a6c-403c-b78e-bab2d20d1c75 admin admin] PATH_INFO: `/v3/users/8f59383e2f6c4da8b683d718f62d25fb` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:27.743172 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685902ff-4a6c-403c-b78e-bab2d20d1c75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f59383e2f6c4da8b683d718f62d25fb)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:27.745133 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685902ff-4a6c-403c-b78e-bab2d20d1c75 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:27.791291 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-685902ff-4a6c-403c-b78e-bab2d20d1c75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f59383e2f6c4da8b683d718f62d25fb'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:27.791950 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-685902ff-4a6c-403c-b78e-bab2d20d1c75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f59383e2f6c4da8b683d718f62d25fb'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:27.804323 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-685902ff-4a6c-403c-b78e-bab2d20d1c75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f59383e2f6c4da8b683d718f62d25fb'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:27.812145 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-685902ff-4a6c-403c-b78e-bab2d20d1c75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f59383e2f6c4da8b683d718f62d25fb'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:27.812882 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-685902ff-4a6c-403c-b78e-bab2d20d1c75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f59383e2f6c4da8b683d718f62d25fb'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:27.827045 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1350/2724] 10.222.0.102 () {58 vars in 1229 bytes} [Thu Sep 1 17:44:27 2022] DELETE /identity/v3/users/8f59383e2f6c4da8b683d718f62d25fb => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:44:28.002584 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c05c9b-ec51-41b5-ad2c-c3441f45e7e8 service placement] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:28.014712 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c05c9b-ec51-41b5-ad2c-c3441f45e7e8 service placement] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:28.072275 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60edb69-338d-4cfe-ba3e-5ff32fac51a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:28.072910 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60edb69-338d-4cfe-ba3e-5ff32fac51a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:28.073192 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60edb69-338d-4cfe-ba3e-5ff32fac51a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:28.073505 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60edb69-338d-4cfe-ba3e-5ff32fac51a8 admin admin] PATH_INFO: `/v3/projects/2cc300abf09e4bb3857336df552286e2` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:28.084238 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60edb69-338d-4cfe-ba3e-5ff32fac51a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2cc300abf09e4bb3857336df552286e2)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:28.086196 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60edb69-338d-4cfe-ba3e-5ff32fac51a8 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:28.132960 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-d60edb69-338d-4cfe-ba3e-5ff32fac51a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2cc300abf09e4bb3857336df552286e2', 'request_id': 'req-d60edb69-338d-4cfe-ba3e-5ff32fac51a8'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:28.139601 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-d60edb69-338d-4cfe-ba3e-5ff32fac51a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2cc300abf09e4bb3857336df552286e2', 'request_id': 'req-d60edb69-338d-4cfe-ba3e-5ff32fac51a8'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:28.146563 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-d60edb69-338d-4cfe-ba3e-5ff32fac51a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2cc300abf09e4bb3857336df552286e2', 'request_id': 'req-d60edb69-338d-4cfe-ba3e-5ff32fac51a8'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:28.158361 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-d60edb69-338d-4cfe-ba3e-5ff32fac51a8 admin admin] The token cache is being invalidate because project 2cc300abf09e4bb3857336df552286e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:44:28.158653 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-d60edb69-338d-4cfe-ba3e-5ff32fac51a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:28.159319 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-d60edb69-338d-4cfe-ba3e-5ff32fac51a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:28.162049 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1375/2725] 10.222.0.102 () {58 vars in 1238 bytes} [Thu Sep 1 17:44:28 2022] DELETE /identity/v3/projects/2cc300abf09e4bb3857336df552286e2 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:44:28.164722 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685902ff-4a6c-403c-b78e-bab2d20d1c75 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:28.166176 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-685902ff-4a6c-403c-b78e-bab2d20d1c75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:28.180444 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685902ff-4a6c-403c-b78e-bab2d20d1c75 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:28.239553 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0443d8e-0fcd-49c6-9028-6ada1deb6584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:28.240193 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0443d8e-0fcd-49c6-9028-6ada1deb6584 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:28.240474 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0443d8e-0fcd-49c6-9028-6ada1deb6584 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:28.240748 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0443d8e-0fcd-49c6-9028-6ada1deb6584 admin admin] PATH_INFO: `/v3/users/04a3973bdbb3406f90d28b3f87c3b577` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:28.253865 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0443d8e-0fcd-49c6-9028-6ada1deb6584 admin admin] RBAC: Authorizing `identity:delete_user(user_id=04a3973bdbb3406f90d28b3f87c3b577)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:28.255813 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0443d8e-0fcd-49c6-9028-6ada1deb6584 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:28.292888 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:28.307106 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-b0443d8e-0fcd-49c6-9028-6ada1deb6584 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04a3973bdbb3406f90d28b3f87c3b577'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:28.307916 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-b0443d8e-0fcd-49c6-9028-6ada1deb6584 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04a3973bdbb3406f90d28b3f87c3b577'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:28.320798 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-b0443d8e-0fcd-49c6-9028-6ada1deb6584 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04a3973bdbb3406f90d28b3f87c3b577'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:28.328497 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-b0443d8e-0fcd-49c6-9028-6ada1deb6584 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04a3973bdbb3406f90d28b3f87c3b577'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:28.329831 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-b0443d8e-0fcd-49c6-9028-6ada1deb6584 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04a3973bdbb3406f90d28b3f87c3b577'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:28.342012 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1351/2726] 10.222.0.102 () {58 vars in 1229 bytes} [Thu Sep 1 17:44:28 2022] DELETE /identity/v3/users/04a3973bdbb3406f90d28b3f87c3b577 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:44:28.515876 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60edb69-338d-4cfe-ba3e-5ff32fac51a8 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:28.517870 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-d60edb69-338d-4cfe-ba3e-5ff32fac51a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:28.535422 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60edb69-338d-4cfe-ba3e-5ff32fac51a8 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:28.591584 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552823f7-2695-4e85-9ab8-d4110a0a6b76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:28.592212 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552823f7-2695-4e85-9ab8-d4110a0a6b76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:28.592494 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552823f7-2695-4e85-9ab8-d4110a0a6b76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:28.592770 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552823f7-2695-4e85-9ab8-d4110a0a6b76 admin admin] PATH_INFO: `/v3/projects/3833a70bee9f499d82a015a856af5107` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:28.602433 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:28.605127 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552823f7-2695-4e85-9ab8-d4110a0a6b76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3833a70bee9f499d82a015a856af5107)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:28.607613 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552823f7-2695-4e85-9ab8-d4110a0a6b76 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:28.654536 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-552823f7-2695-4e85-9ab8-d4110a0a6b76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3833a70bee9f499d82a015a856af5107', 'request_id': 'req-552823f7-2695-4e85-9ab8-d4110a0a6b76'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:28.661560 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-552823f7-2695-4e85-9ab8-d4110a0a6b76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3833a70bee9f499d82a015a856af5107', 'request_id': 'req-552823f7-2695-4e85-9ab8-d4110a0a6b76'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:28.668574 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-552823f7-2695-4e85-9ab8-d4110a0a6b76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3833a70bee9f499d82a015a856af5107', 'request_id': 'req-552823f7-2695-4e85-9ab8-d4110a0a6b76'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:28.680235 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-552823f7-2695-4e85-9ab8-d4110a0a6b76 admin admin] The token cache is being invalidate because project 3833a70bee9f499d82a015a856af5107 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:44:28.680532 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-552823f7-2695-4e85-9ab8-d4110a0a6b76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:28.681326 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-552823f7-2695-4e85-9ab8-d4110a0a6b76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:28.682849 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1376/2727] 10.222.0.102 () {58 vars in 1238 bytes} [Thu Sep 1 17:44:28 2022] DELETE /identity/v3/projects/3833a70bee9f499d82a015a856af5107 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:44:38.338557 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=18,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:38.676890 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=25,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:47.194564 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0443d8e-0fcd-49c6-9028-6ada1deb6584 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:47.196058 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-b0443d8e-0fcd-49c6-9028-6ada1deb6584 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:47.269897 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0443d8e-0fcd-49c6-9028-6ada1deb6584 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:47.332777 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e3d1c2-34e8-4f1b-a97a-adcd4181df60 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f8293316baf4774b62fc11d55d50865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:47.333951 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e3d1c2-34e8-4f1b-a97a-adcd4181df60 service nova] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:47.334229 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e3d1c2-34e8-4f1b-a97a-adcd4181df60 service nova] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:47.334492 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e3d1c2-34e8-4f1b-a97a-adcd4181df60 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:47.336225 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-75e3d1c2-34e8-4f1b-a97a-adcd4181df60 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:47.348979 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e3d1c2-34e8-4f1b-a97a-adcd4181df60 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:47.350921 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e3d1c2-34e8-4f1b-a97a-adcd4181df60 service nova] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:47.362498 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1352/2728] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:44:47 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:44:48.042813 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552823f7-2695-4e85-9ab8-d4110a0a6b76 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:48.044743 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-552823f7-2695-4e85-9ab8-d4110a0a6b76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:48.116284 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552823f7-2695-4e85-9ab8-d4110a0a6b76 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:48.122403 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19327615-e9a6-46a9-8283-2975fbcd9e10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:48.123047 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19327615-e9a6-46a9-8283-2975fbcd9e10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:48.123324 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19327615-e9a6-46a9-8283-2975fbcd9e10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:48.123628 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19327615-e9a6-46a9-8283-2975fbcd9e10 admin admin] PATH_INFO: `/v3/users/1755e51704664616b77422cc0d49b09f` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:48.124812 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19327615-e9a6-46a9-8283-2975fbcd9e10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1755e51704664616b77422cc0d49b09f)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:48.126772 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19327615-e9a6-46a9-8283-2975fbcd9e10 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:48.175048 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-19327615-e9a6-46a9-8283-2975fbcd9e10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1755e51704664616b77422cc0d49b09f'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:48.187802 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-19327615-e9a6-46a9-8283-2975fbcd9e10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1755e51704664616b77422cc0d49b09f'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:48.195635 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-19327615-e9a6-46a9-8283-2975fbcd9e10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1755e51704664616b77422cc0d49b09f'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:48.196540 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-19327615-e9a6-46a9-8283-2975fbcd9e10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1755e51704664616b77422cc0d49b09f'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:48.197196 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-19327615-e9a6-46a9-8283-2975fbcd9e10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1755e51704664616b77422cc0d49b09f'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:48.209472 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1377/2729] 10.222.0.102 () {58 vars in 1229 bytes} [Thu Sep 1 17:44:48 2022] DELETE /identity/v3/users/1755e51704664616b77422cc0d49b09f => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:44:48.213926 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bfbb56-8907-4b29-a624-d9577ee11eb1 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:48.214418 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bfbb56-8907-4b29-a624-d9577ee11eb1 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:48.214530 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bfbb56-8907-4b29-a624-d9577ee11eb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:48.234046 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-97bfbb56-8907-4b29-a624-d9577ee11eb1 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:48.235259 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-97bfbb56-8907-4b29-a624-d9577ee11eb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:48.295239 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1353/2730] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:44:48 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:44:48.306216 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19327615-e9a6-46a9-8283-2975fbcd9e10 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:48.307615 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-19327615-e9a6-46a9-8283-2975fbcd9e10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:48.364163 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19327615-e9a6-46a9-8283-2975fbcd9e10 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:48.368334 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70433ffb-cf46-41fd-8307-f4f937d2bc18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:48.368925 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70433ffb-cf46-41fd-8307-f4f937d2bc18 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:48.369199 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70433ffb-cf46-41fd-8307-f4f937d2bc18 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:48.369509 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70433ffb-cf46-41fd-8307-f4f937d2bc18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:48.378275 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70433ffb-cf46-41fd-8307-f4f937d2bc18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:48.379685 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70433ffb-cf46-41fd-8307-f4f937d2bc18 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:48.388870 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1378/2731] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:44:48 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:44:48.657096 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bfbb56-8907-4b29-a624-d9577ee11eb1 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:48.658566 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-97bfbb56-8907-4b29-a624-d9577ee11eb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:48.671096 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bfbb56-8907-4b29-a624-d9577ee11eb1 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:48.676242 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78eebd49-d8df-4a66-be53-4795b4225c30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:48.676869 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eebd49-d8df-4a66-be53-4795b4225c30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:48.677146 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eebd49-d8df-4a66-be53-4795b4225c30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:48.677452 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eebd49-d8df-4a66-be53-4795b4225c30 admin admin] PATH_INFO: `/v3/projects/ed870e25f9d545e8b027fe724e6eb285` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:48.678564 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78eebd49-d8df-4a66-be53-4795b4225c30 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed870e25f9d545e8b027fe724e6eb285)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:48.680479 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78eebd49-d8df-4a66-be53-4795b4225c30 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:48.702468 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:48.729732 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-78eebd49-d8df-4a66-be53-4795b4225c30 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed870e25f9d545e8b027fe724e6eb285', 'request_id': 'req-78eebd49-d8df-4a66-be53-4795b4225c30'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:48.735495 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-78eebd49-d8df-4a66-be53-4795b4225c30 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed870e25f9d545e8b027fe724e6eb285', 'request_id': 'req-78eebd49-d8df-4a66-be53-4795b4225c30'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:48.742344 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-78eebd49-d8df-4a66-be53-4795b4225c30 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed870e25f9d545e8b027fe724e6eb285', 'request_id': 'req-78eebd49-d8df-4a66-be53-4795b4225c30'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:48.753536 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-78eebd49-d8df-4a66-be53-4795b4225c30 admin admin] The token cache is being invalidate because project ed870e25f9d545e8b027fe724e6eb285 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:44:48.753828 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-78eebd49-d8df-4a66-be53-4795b4225c30 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:48.754453 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-78eebd49-d8df-4a66-be53-4795b4225c30 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:48.755926 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1354/2732] 10.222.0.102 () {58 vars in 1238 bytes} [Thu Sep 1 17:44:48 2022] DELETE /identity/v3/projects/ed870e25f9d545e8b027fe724e6eb285 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:44:48.760097 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70433ffb-cf46-41fd-8307-f4f937d2bc18 service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:48.769923 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70433ffb-cf46-41fd-8307-f4f937d2bc18 service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:48.789574 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:48.830395 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2041365c-9a3b-4be3-931a-600d9c4b0667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:48.831026 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2041365c-9a3b-4be3-931a-600d9c4b0667 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:48.831308 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2041365c-9a3b-4be3-931a-600d9c4b0667 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:48.831581 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2041365c-9a3b-4be3-931a-600d9c4b0667 admin admin] PATH_INFO: `/v3/users/c8ac3f780ca34123a1372bea4a4cebcb` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:48.832695 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2041365c-9a3b-4be3-931a-600d9c4b0667 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8ac3f780ca34123a1372bea4a4cebcb)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:48.835286 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2041365c-9a3b-4be3-931a-600d9c4b0667 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:48.879815 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-2041365c-9a3b-4be3-931a-600d9c4b0667 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8ac3f780ca34123a1372bea4a4cebcb'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:48.891892 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-2041365c-9a3b-4be3-931a-600d9c4b0667 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8ac3f780ca34123a1372bea4a4cebcb'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:48.899579 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-2041365c-9a3b-4be3-931a-600d9c4b0667 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8ac3f780ca34123a1372bea4a4cebcb'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:48.900176 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-2041365c-9a3b-4be3-931a-600d9c4b0667 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8ac3f780ca34123a1372bea4a4cebcb'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:48.900781 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-2041365c-9a3b-4be3-931a-600d9c4b0667 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8ac3f780ca34123a1372bea4a4cebcb'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:48.912840 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1379/2733] 10.222.0.102 () {58 vars in 1229 bytes} [Thu Sep 1 17:44:48 2022] DELETE /identity/v3/users/c8ac3f780ca34123a1372bea4a4cebcb => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:44:49.188404 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78eebd49-d8df-4a66-be53-4795b4225c30 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:49.190028 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-78eebd49-d8df-4a66-be53-4795b4225c30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:49.206021 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78eebd49-d8df-4a66-be53-4795b4225c30 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:49.265462 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a000914-9b28-4af2-b0ac-14b0ee1ef109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:49.266131 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a000914-9b28-4af2-b0ac-14b0ee1ef109 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:49.266361 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a000914-9b28-4af2-b0ac-14b0ee1ef109 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:49.266634 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a000914-9b28-4af2-b0ac-14b0ee1ef109 admin admin] PATH_INFO: `/v3/projects/c26adac2e00949cfa1c27e475a12e79d` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:49.267798 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a000914-9b28-4af2-b0ac-14b0ee1ef109 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c26adac2e00949cfa1c27e475a12e79d)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:49.269748 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a000914-9b28-4af2-b0ac-14b0ee1ef109 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:49.316512 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-5a000914-9b28-4af2-b0ac-14b0ee1ef109 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c26adac2e00949cfa1c27e475a12e79d', 'request_id': 'req-5a000914-9b28-4af2-b0ac-14b0ee1ef109'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:49.322998 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-5a000914-9b28-4af2-b0ac-14b0ee1ef109 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c26adac2e00949cfa1c27e475a12e79d', 'request_id': 'req-5a000914-9b28-4af2-b0ac-14b0ee1ef109'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:49.330381 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-5a000914-9b28-4af2-b0ac-14b0ee1ef109 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c26adac2e00949cfa1c27e475a12e79d', 'request_id': 'req-5a000914-9b28-4af2-b0ac-14b0ee1ef109'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:49.341288 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-5a000914-9b28-4af2-b0ac-14b0ee1ef109 admin admin] The token cache is being invalidate because project c26adac2e00949cfa1c27e475a12e79d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:44:49.341639 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-5a000914-9b28-4af2-b0ac-14b0ee1ef109 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:49.342285 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-5a000914-9b28-4af2-b0ac-14b0ee1ef109 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:49.343759 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1355/2734] 10.222.0.102 () {58 vars in 1238 bytes} [Thu Sep 1 17:44:49 2022] DELETE /identity/v3/projects/c26adac2e00949cfa1c27e475a12e79d => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:44:49.348741 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a22414-65d1-4eba-b632-9b1651c528ad None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:49.349322 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a22414-65d1-4eba-b632-9b1651c528ad None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:49.349424 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a22414-65d1-4eba-b632-9b1651c528ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:49.370603 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-47a22414-65d1-4eba-b632-9b1651c528ad None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:49.371803 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-47a22414-65d1-4eba-b632-9b1651c528ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:49.429182 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1380/2735] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:44:49 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:44:49.437114 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad97bf1-e2ce-499a-8cc5-afa1187a96cc None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:49.437428 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad97bf1-e2ce-499a-8cc5-afa1187a96cc None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:49.437705 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad97bf1-e2ce-499a-8cc5-afa1187a96cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:49.457621 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-5ad97bf1-e2ce-499a-8cc5-afa1187a96cc None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:49.459098 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-5ad97bf1-e2ce-499a-8cc5-afa1187a96cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:49.473235 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1356/2736] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:44:49 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:44:49.477715 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a22414-65d1-4eba-b632-9b1651c528ad None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:49.487669 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a22414-65d1-4eba-b632-9b1651c528ad None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:49.492831 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1987c3-02e2-455a-9066-6faf6ccf882c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:49.493476 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1987c3-02e2-455a-9066-6faf6ccf882c admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:49.493756 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1987c3-02e2-455a-9066-6faf6ccf882c admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:49.494058 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1987c3-02e2-455a-9066-6faf6ccf882c admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:49.494711 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1987c3-02e2-455a-9066-6faf6ccf882c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:44:49.495017 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1987c3-02e2-455a-9066-6faf6ccf882c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:49.496618 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1987c3-02e2-455a-9066-6faf6ccf882c admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:49.509111 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1381/2737] 10.222.0.102 () {58 vars in 1158 bytes} [Thu Sep 1 17:44:49 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:49.513572 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad97bf1-e2ce-499a-8cc5-afa1187a96cc None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:49.522859 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad97bf1-e2ce-499a-8cc5-afa1187a96cc None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:49.527496 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bc7571-abe0-4c31-a914-8332ce74a947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:49.528086 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bc7571-abe0-4c31-a914-8332ce74a947 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:49.528366 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bc7571-abe0-4c31-a914-8332ce74a947 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:49.528639 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bc7571-abe0-4c31-a914-8332ce74a947 admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:49.529268 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bc7571-abe0-4c31-a914-8332ce74a947 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:49.531192 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bc7571-abe0-4c31-a914-8332ce74a947 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:49.561216 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1357/2738] 10.222.0.102 () {60 vars in 1158 bytes} [Thu Sep 1 17:44:49 2022] POST /identity/v3/projects => generated 398 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:44:49.562751 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1987c3-02e2-455a-9066-6faf6ccf882c admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:49.573351 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1987c3-02e2-455a-9066-6faf6ccf882c admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:49.598642 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:49.631544 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8f9873-8b8f-4542-9bd3-edac9bb049e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:49.632140 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8f9873-8b8f-4542-9bd3-edac9bb049e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:49.632420 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8f9873-8b8f-4542-9bd3-edac9bb049e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:49.632739 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8f9873-8b8f-4542-9bd3-edac9bb049e4 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:49.633498 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8f9873-8b8f-4542-9bd3-edac9bb049e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:49.635421 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8f9873-8b8f-4542-9bd3-edac9bb049e4 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:49.656642 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1382/2739] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:44:49 2022] POST /identity/v3/users => generated 300 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:44:49.660774 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bc7571-abe0-4c31-a914-8332ce74a947 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:49.671636 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bc7571-abe0-4c31-a914-8332ce74a947 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:49.676145 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e0e48f-b2d8-4fd1-a8d2-fd366b807ac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:49.676727 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e0e48f-b2d8-4fd1-a8d2-fd366b807ac3 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:49.677642 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e0e48f-b2d8-4fd1-a8d2-fd366b807ac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:49.677917 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e0e48f-b2d8-4fd1-a8d2-fd366b807ac3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:49.678543 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e0e48f-b2d8-4fd1-a8d2-fd366b807ac3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:49.680123 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e0e48f-b2d8-4fd1-a8d2-fd366b807ac3 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:49.682120 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:49.691501 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1358/2740] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:44:49 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:44:49.695154 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8f9873-8b8f-4542-9bd3-edac9bb049e4 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:49.705323 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8f9873-8b8f-4542-9bd3-edac9bb049e4 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:49.709884 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a5e87d-8566-4593-be5c-99a6d13bebba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:49.710471 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a5e87d-8566-4593-be5c-99a6d13bebba admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:49.710751 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a5e87d-8566-4593-be5c-99a6d13bebba admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:49.711025 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a5e87d-8566-4593-be5c-99a6d13bebba admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:49.711661 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a5e87d-8566-4593-be5c-99a6d13bebba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:49.713660 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a5e87d-8566-4593-be5c-99a6d13bebba admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:49.724819 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1383/2741] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:44:49 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:44:49.728970 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e0e48f-b2d8-4fd1-a8d2-fd366b807ac3 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:49.738809 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e0e48f-b2d8-4fd1-a8d2-fd366b807ac3 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:49.743965 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09652a70-a0c3-4884-baf9-297161df9252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:49.744608 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09652a70-a0c3-4884-baf9-297161df9252 admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:49.744922 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09652a70-a0c3-4884-baf9-297161df9252 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:49.745218 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09652a70-a0c3-4884-baf9-297161df9252 admin admin] PATH_INFO: `/v3/projects/fea994039b4c431d929a19bb863beb56/users/fdd0e65035684a8aacf570dec42c586b/roles/b75cb360925e4ba5a65f14965c103ffd` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:49.759819 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09652a70-a0c3-4884-baf9-297161df9252 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fea994039b4c431d929a19bb863beb56, user_id=fdd0e65035684a8aacf570dec42c586b, role_id=b75cb360925e4ba5a65f14965c103ffd)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:49.763566 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09652a70-a0c3-4884-baf9-297161df9252 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:49.777321 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1359/2742] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:44:49 2022] PUT /identity/v3/projects/fea994039b4c431d929a19bb863beb56/users/fdd0e65035684a8aacf570dec42c586b/roles/b75cb360925e4ba5a65f14965c103ffd => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:44:49.789181 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6141481b-c4e5-4ab0-a911-02ecd52b7df7 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:49.789599 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6141481b-c4e5-4ab0-a911-02ecd52b7df7 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:49.789878 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6141481b-c4e5-4ab0-a911-02ecd52b7df7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:49.818275 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-6141481b-c4e5-4ab0-a911-02ecd52b7df7 None None] MFA Rules not processed for user `fdd0e65035684a8aacf570dec42c586b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:49.819515 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-6141481b-c4e5-4ab0-a911-02ecd52b7df7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:49.884499 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1384/2743] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:44:49 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:44:49.888940 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09652a70-a0c3-4884-baf9-297161df9252 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:49.949775 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09652a70-a0c3-4884-baf9-297161df9252 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:49.954377 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca0e66a-ef5e-45c6-9e3a-8927fb773a96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:49.954971 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca0e66a-ef5e-45c6-9e3a-8927fb773a96 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:49.955248 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca0e66a-ef5e-45c6-9e3a-8927fb773a96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:49.955516 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca0e66a-ef5e-45c6-9e3a-8927fb773a96 admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:49.956141 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca0e66a-ef5e-45c6-9e3a-8927fb773a96 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:49.958047 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca0e66a-ef5e-45c6-9e3a-8927fb773a96 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:49.988272 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1360/2744] 10.222.0.102 () {60 vars in 1158 bytes} [Thu Sep 1 17:44:49 2022] POST /identity/v3/projects => generated 402 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:44:49.989377 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6141481b-c4e5-4ab0-a911-02ecd52b7df7 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:50.000065 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6141481b-c4e5-4ab0-a911-02ecd52b7df7 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:50.056424 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77339a3-bfd4-4dc6-9fae-a78e26db2ab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:50.057027 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77339a3-bfd4-4dc6-9fae-a78e26db2ab7 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:50.057327 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77339a3-bfd4-4dc6-9fae-a78e26db2ab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:50.057636 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77339a3-bfd4-4dc6-9fae-a78e26db2ab7 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:50.058277 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77339a3-bfd4-4dc6-9fae-a78e26db2ab7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:50.060294 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77339a3-bfd4-4dc6-9fae-a78e26db2ab7 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:50.081557 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1385/2745] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:44:49 2022] POST /identity/v3/users => generated 308 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:44:50.086071 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca0e66a-ef5e-45c6-9e3a-8927fb773a96 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:50.096551 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca0e66a-ef5e-45c6-9e3a-8927fb773a96 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:50.102114 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c48c44d-efed-4829-81e0-56599e1d1cda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:50.102721 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c48c44d-efed-4829-81e0-56599e1d1cda admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:50.103004 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c48c44d-efed-4829-81e0-56599e1d1cda admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:50.103314 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c48c44d-efed-4829-81e0-56599e1d1cda admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:50.103978 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c48c44d-efed-4829-81e0-56599e1d1cda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:50.105666 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c48c44d-efed-4829-81e0-56599e1d1cda admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:50.117196 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1361/2746] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:44:50 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 17:44:50.121224 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77339a3-bfd4-4dc6-9fae-a78e26db2ab7 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:50.131101 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77339a3-bfd4-4dc6-9fae-a78e26db2ab7 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:50.136055 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b73def1-4d31-407b-a8e8-30ea138c0ac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:50.136667 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b73def1-4d31-407b-a8e8-30ea138c0ac8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:50.136951 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b73def1-4d31-407b-a8e8-30ea138c0ac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:50.137228 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b73def1-4d31-407b-a8e8-30ea138c0ac8 admin admin] PATH_INFO: `/v3/projects/da42bcb5d408476c983e2caaee44356e/users/9f1c17333a084616b9a1e8f005234467/roles/36577a6f46d642d4b81805ede9872490` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:50.152917 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b73def1-4d31-407b-a8e8-30ea138c0ac8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da42bcb5d408476c983e2caaee44356e, user_id=9f1c17333a084616b9a1e8f005234467, role_id=36577a6f46d642d4b81805ede9872490)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:50.155640 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b73def1-4d31-407b-a8e8-30ea138c0ac8 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:50.168653 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1386/2747] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:44:50 2022] PUT /identity/v3/projects/da42bcb5d408476c983e2caaee44356e/users/9f1c17333a084616b9a1e8f005234467/roles/36577a6f46d642d4b81805ede9872490 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:44:50.172597 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c48c44d-efed-4829-81e0-56599e1d1cda admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:50.182535 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c48c44d-efed-4829-81e0-56599e1d1cda admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:50.237627 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2f8239-e331-46b1-917b-56e1b423dca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:50.238229 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2f8239-e331-46b1-917b-56e1b423dca3 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:50.238512 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2f8239-e331-46b1-917b-56e1b423dca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:50.238807 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2f8239-e331-46b1-917b-56e1b423dca3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:50.239982 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2f8239-e331-46b1-917b-56e1b423dca3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:50.241667 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2f8239-e331-46b1-917b-56e1b423dca3 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:50.251584 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1362/2748] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:44:50 2022] GET /identity/v3/roles => generated 1427 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:44:50.256491 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b73def1-4d31-407b-a8e8-30ea138c0ac8 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:50.261950 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:50.268077 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b73def1-4d31-407b-a8e8-30ea138c0ac8 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:50.272971 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75db178d-092b-4d26-9615-3e0644b519ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:50.273673 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75db178d-092b-4d26-9615-3e0644b519ec admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:50.273989 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75db178d-092b-4d26-9615-3e0644b519ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:50.274304 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75db178d-092b-4d26-9615-3e0644b519ec admin admin] PATH_INFO: `/v3/projects/da42bcb5d408476c983e2caaee44356e/users/9f1c17333a084616b9a1e8f005234467/roles/36577a6f46d642d4b81805ede9872490` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:50.276293 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75db178d-092b-4d26-9615-3e0644b519ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=da42bcb5d408476c983e2caaee44356e, user_id=9f1c17333a084616b9a1e8f005234467, role_id=36577a6f46d642d4b81805ede9872490)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:50.279081 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75db178d-092b-4d26-9615-3e0644b519ec admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:50.289885 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1387/2749] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:44:50 2022] PUT /identity/v3/projects/da42bcb5d408476c983e2caaee44356e/users/9f1c17333a084616b9a1e8f005234467/roles/36577a6f46d642d4b81805ede9872490 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:44:50.300123 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab01e521-359a-4d3f-9bbb-dad824b729f4 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:50.300592 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab01e521-359a-4d3f-9bbb-dad824b729f4 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:50.300712 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab01e521-359a-4d3f-9bbb-dad824b729f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:50.328375 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-ab01e521-359a-4d3f-9bbb-dad824b729f4 None None] MFA Rules not processed for user `9f1c17333a084616b9a1e8f005234467`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:50.329595 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-ab01e521-359a-4d3f-9bbb-dad824b729f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:50.361999 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:50.398379 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1363/2750] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:44:50 2022] POST /identity/v3/auth/tokens => generated 3355 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:44:50.408791 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75db178d-092b-4d26-9615-3e0644b519ec admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:50.410532 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-75db178d-092b-4d26-9615-3e0644b519ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:50.468230 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75db178d-092b-4d26-9615-3e0644b519ec admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:50.471896 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e6aee6-ce72-4a4c-a1ed-9b5d7c0918ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '523ed589872a45a09b6a7675a7fb0b75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:50.472518 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e6aee6-ce72-4a4c-a1ed-9b5d7c0918ac service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:50.472799 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e6aee6-ce72-4a4c-a1ed-9b5d7c0918ac service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:50.473071 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e6aee6-ce72-4a4c-a1ed-9b5d7c0918ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:50.482738 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e6aee6-ce72-4a4c-a1ed-9b5d7c0918ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:50.484235 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e6aee6-ce72-4a4c-a1ed-9b5d7c0918ac service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:50.494160 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1388/2751] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:44:50 2022] GET /identity/v3/auth/tokens => generated 3355 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:44:50.570886 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab01e521-359a-4d3f-9bbb-dad824b729f4 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:50.580393 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab01e521-359a-4d3f-9bbb-dad824b729f4 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:50.584124 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a921bb3-f21f-4cdd-8dbc-dc3ea29dca83 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '523ed589872a45a09b6a7675a7fb0b75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:50.584719 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a921bb3-f21f-4cdd-8dbc-dc3ea29dca83 service cinder] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:50.584997 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a921bb3-f21f-4cdd-8dbc-dc3ea29dca83 service cinder] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:50.585267 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a921bb3-f21f-4cdd-8dbc-dc3ea29dca83 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:50.641981 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a921bb3-f21f-4cdd-8dbc-dc3ea29dca83 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:50.643485 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a921bb3-f21f-4cdd-8dbc-dc3ea29dca83 service cinder] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:50.652350 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1364/2752] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:44:50 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 17:44:54.692200 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e6aee6-ce72-4a4c-a1ed-9b5d7c0918ac service cinder] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:54.752229 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e6aee6-ce72-4a4c-a1ed-9b5d7c0918ac service cinder] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:54.757324 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d0fe90-17f8-41bb-89b6-4a63b27642d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:54.757954 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d0fe90-17f8-41bb-89b6-4a63b27642d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:54.758232 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d0fe90-17f8-41bb-89b6-4a63b27642d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:54.758505 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d0fe90-17f8-41bb-89b6-4a63b27642d4 admin admin] PATH_INFO: `/v3/users/fdd0e65035684a8aacf570dec42c586b` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:54.759606 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d0fe90-17f8-41bb-89b6-4a63b27642d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fdd0e65035684a8aacf570dec42c586b)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:54.761551 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d0fe90-17f8-41bb-89b6-4a63b27642d4 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:54.806402 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b9d0fe90-17f8-41bb-89b6-4a63b27642d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fdd0e65035684a8aacf570dec42c586b'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:54.820317 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b9d0fe90-17f8-41bb-89b6-4a63b27642d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fdd0e65035684a8aacf570dec42c586b'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:54.827272 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b9d0fe90-17f8-41bb-89b6-4a63b27642d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fdd0e65035684a8aacf570dec42c586b'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:54.827998 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b9d0fe90-17f8-41bb-89b6-4a63b27642d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fdd0e65035684a8aacf570dec42c586b'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:54.828614 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b9d0fe90-17f8-41bb-89b6-4a63b27642d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fdd0e65035684a8aacf570dec42c586b'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:54.840794 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1389/2753] 10.222.0.102 () {58 vars in 1229 bytes} [Thu Sep 1 17:44:54 2022] DELETE /identity/v3/users/fdd0e65035684a8aacf570dec42c586b => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:44:54.845461 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe7cd75-8b6d-45e6-93c4-bdb4de3c778f None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:54.845928 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe7cd75-8b6d-45e6-93c4-bdb4de3c778f None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:54.846025 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe7cd75-8b6d-45e6-93c4-bdb4de3c778f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:54.867155 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-bbe7cd75-8b6d-45e6-93c4-bdb4de3c778f None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:54.868686 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-bbe7cd75-8b6d-45e6-93c4-bdb4de3c778f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:54.929788 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1365/2754] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:44:54 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:44:54.939562 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d0fe90-17f8-41bb-89b6-4a63b27642d4 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:54.940991 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-b9d0fe90-17f8-41bb-89b6-4a63b27642d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:54.954921 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:55.001348 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d0fe90-17f8-41bb-89b6-4a63b27642d4 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:55.005001 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b5c769-7b1a-4826-bc06-66890f50a157 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:55.005675 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b5c769-7b1a-4826-bc06-66890f50a157 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:55.006015 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b5c769-7b1a-4826-bc06-66890f50a157 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:55.006193 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b5c769-7b1a-4826-bc06-66890f50a157 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:55.015010 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b5c769-7b1a-4826-bc06-66890f50a157 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:55.016433 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b5c769-7b1a-4826-bc06-66890f50a157 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:55.026011 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1390/2755] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:44:54 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:44:55.292396 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe7cd75-8b6d-45e6-93c4-bdb4de3c778f None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:55.293945 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-bbe7cd75-8b6d-45e6-93c4-bdb4de3c778f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:55.306333 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe7cd75-8b6d-45e6-93c4-bdb4de3c778f None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:55.311176 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a221d600-2d24-4e41-b3b5-64bb476ac823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:55.311805 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a221d600-2d24-4e41-b3b5-64bb476ac823 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:55.312088 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a221d600-2d24-4e41-b3b5-64bb476ac823 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:55.312417 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a221d600-2d24-4e41-b3b5-64bb476ac823 admin admin] PATH_INFO: `/v3/projects/fea994039b4c431d929a19bb863beb56` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:55.313522 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a221d600-2d24-4e41-b3b5-64bb476ac823 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fea994039b4c431d929a19bb863beb56)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:55.315441 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a221d600-2d24-4e41-b3b5-64bb476ac823 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:55.366146 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-a221d600-2d24-4e41-b3b5-64bb476ac823 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fea994039b4c431d929a19bb863beb56', 'request_id': 'req-a221d600-2d24-4e41-b3b5-64bb476ac823'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:55.371796 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-a221d600-2d24-4e41-b3b5-64bb476ac823 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fea994039b4c431d929a19bb863beb56', 'request_id': 'req-a221d600-2d24-4e41-b3b5-64bb476ac823'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:55.375245 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:55.379522 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-a221d600-2d24-4e41-b3b5-64bb476ac823 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fea994039b4c431d929a19bb863beb56', 'request_id': 'req-a221d600-2d24-4e41-b3b5-64bb476ac823'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:55.391851 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-a221d600-2d24-4e41-b3b5-64bb476ac823 admin admin] The token cache is being invalidate because project fea994039b4c431d929a19bb863beb56 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:44:55.392160 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-a221d600-2d24-4e41-b3b5-64bb476ac823 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:55.392763 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-a221d600-2d24-4e41-b3b5-64bb476ac823 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:55.394264 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1366/2756] 10.222.0.102 () {58 vars in 1238 bytes} [Thu Sep 1 17:44:55 2022] DELETE /identity/v3/projects/fea994039b4c431d929a19bb863beb56 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:44:55.398814 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b5c769-7b1a-4826-bc06-66890f50a157 service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:55.408220 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b5c769-7b1a-4826-bc06-66890f50a157 service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:55.463607 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc96f760-7b82-4876-a81b-2a4a4f191c81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:55.464228 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc96f760-7b82-4876-a81b-2a4a4f191c81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:55.464508 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc96f760-7b82-4876-a81b-2a4a4f191c81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:55.464783 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc96f760-7b82-4876-a81b-2a4a4f191c81 admin admin] PATH_INFO: `/v3/users/9f1c17333a084616b9a1e8f005234467` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:55.465914 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc96f760-7b82-4876-a81b-2a4a4f191c81 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f1c17333a084616b9a1e8f005234467)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:55.467849 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc96f760-7b82-4876-a81b-2a4a4f191c81 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:55.511338 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-fc96f760-7b82-4876-a81b-2a4a4f191c81 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f1c17333a084616b9a1e8f005234467'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:55.523371 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-fc96f760-7b82-4876-a81b-2a4a4f191c81 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f1c17333a084616b9a1e8f005234467'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:55.529713 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-fc96f760-7b82-4876-a81b-2a4a4f191c81 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f1c17333a084616b9a1e8f005234467'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:55.530322 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-fc96f760-7b82-4876-a81b-2a4a4f191c81 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f1c17333a084616b9a1e8f005234467'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:55.530944 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-fc96f760-7b82-4876-a81b-2a4a4f191c81 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f1c17333a084616b9a1e8f005234467'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:55.543451 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1391/2757] 10.222.0.102 () {58 vars in 1229 bytes} [Thu Sep 1 17:44:55 2022] DELETE /identity/v3/users/9f1c17333a084616b9a1e8f005234467 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:44:55.806990 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a221d600-2d24-4e41-b3b5-64bb476ac823 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:55.809511 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-a221d600-2d24-4e41-b3b5-64bb476ac823 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:55.823092 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a221d600-2d24-4e41-b3b5-64bb476ac823 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:55.902205 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e3e06a-e35b-4c3f-ab3a-87d4b119bdc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:55.902882 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e3e06a-e35b-4c3f-ab3a-87d4b119bdc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:55.903047 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e3e06a-e35b-4c3f-ab3a-87d4b119bdc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:55.903323 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e3e06a-e35b-4c3f-ab3a-87d4b119bdc4 admin admin] PATH_INFO: `/v3/projects/da42bcb5d408476c983e2caaee44356e` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:55.904517 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e3e06a-e35b-4c3f-ab3a-87d4b119bdc4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da42bcb5d408476c983e2caaee44356e)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:55.908372 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e3e06a-e35b-4c3f-ab3a-87d4b119bdc4 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:55.957699 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-d4e3e06a-e35b-4c3f-ab3a-87d4b119bdc4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da42bcb5d408476c983e2caaee44356e', 'request_id': 'req-d4e3e06a-e35b-4c3f-ab3a-87d4b119bdc4'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:55.963208 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-d4e3e06a-e35b-4c3f-ab3a-87d4b119bdc4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da42bcb5d408476c983e2caaee44356e', 'request_id': 'req-d4e3e06a-e35b-4c3f-ab3a-87d4b119bdc4'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:55.970429 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-d4e3e06a-e35b-4c3f-ab3a-87d4b119bdc4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da42bcb5d408476c983e2caaee44356e', 'request_id': 'req-d4e3e06a-e35b-4c3f-ab3a-87d4b119bdc4'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:55.981832 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-d4e3e06a-e35b-4c3f-ab3a-87d4b119bdc4 admin admin] The token cache is being invalidate because project da42bcb5d408476c983e2caaee44356e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:44:55.982120 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-d4e3e06a-e35b-4c3f-ab3a-87d4b119bdc4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:55.982774 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-d4e3e06a-e35b-4c3f-ab3a-87d4b119bdc4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:55.984246 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1367/2758] 10.222.0.102 () {58 vars in 1238 bytes} [Thu Sep 1 17:44:55 2022] DELETE /identity/v3/projects/da42bcb5d408476c983e2caaee44356e => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:44:55.989067 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5c6d37-e0de-4e50-bf32-dc2666de176f None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:55.989551 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5c6d37-e0de-4e50-bf32-dc2666de176f None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:55.989636 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5c6d37-e0de-4e50-bf32-dc2666de176f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:56.010047 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-4b5c6d37-e0de-4e50-bf32-dc2666de176f None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:56.011247 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-4b5c6d37-e0de-4e50-bf32-dc2666de176f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:56.040042 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:56.072743 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1392/2759] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:44:55 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:44:56.080659 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d88b04-1faf-4dd4-9bba-5238254017c9 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:56.080961 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d88b04-1faf-4dd4-9bba-5238254017c9 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:56.081225 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d88b04-1faf-4dd4-9bba-5238254017c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:56.100564 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-56d88b04-1faf-4dd4-9bba-5238254017c9 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:56.102074 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-56d88b04-1faf-4dd4-9bba-5238254017c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:56.109902 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1368/2760] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:44:56 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:44:56.114042 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5c6d37-e0de-4e50-bf32-dc2666de176f None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:56.124207 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5c6d37-e0de-4e50-bf32-dc2666de176f None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:56.128767 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e3ea2b-f17d-4b37-b23e-149910c354c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:56.129388 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e3ea2b-f17d-4b37-b23e-149910c354c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:56.129680 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e3ea2b-f17d-4b37-b23e-149910c354c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:56.129964 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e3ea2b-f17d-4b37-b23e-149910c354c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:56.131187 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e3ea2b-f17d-4b37-b23e-149910c354c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:44:56.131495 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e3ea2b-f17d-4b37-b23e-149910c354c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:56.133092 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e3ea2b-f17d-4b37-b23e-149910c354c7 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:56.144370 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1393/2761] 10.222.0.102 () {58 vars in 1158 bytes} [Thu Sep 1 17:44:56 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:56.148556 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d88b04-1faf-4dd4-9bba-5238254017c9 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:56.157888 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d88b04-1faf-4dd4-9bba-5238254017c9 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:56.162491 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf76d72-11c2-4c1d-8dc7-e77cae87da2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:56.163077 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf76d72-11c2-4c1d-8dc7-e77cae87da2e admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:56.163354 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf76d72-11c2-4c1d-8dc7-e77cae87da2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:56.163623 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf76d72-11c2-4c1d-8dc7-e77cae87da2e admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:56.164250 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf76d72-11c2-4c1d-8dc7-e77cae87da2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:56.166175 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf76d72-11c2-4c1d-8dc7-e77cae87da2e admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:56.193949 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1369/2762] 10.222.0.102 () {60 vars in 1158 bytes} [Thu Sep 1 17:44:56 2022] POST /identity/v3/projects => generated 368 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:44:56.198068 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e3ea2b-f17d-4b37-b23e-149910c354c7 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:56.207751 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e3ea2b-f17d-4b37-b23e-149910c354c7 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:56.262659 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc95ad13-05fc-4189-8777-85de33e5366a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:56.263327 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc95ad13-05fc-4189-8777-85de33e5366a admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:56.263608 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc95ad13-05fc-4189-8777-85de33e5366a admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:56.263880 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc95ad13-05fc-4189-8777-85de33e5366a admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:56.264491 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc95ad13-05fc-4189-8777-85de33e5366a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:56.266439 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc95ad13-05fc-4189-8777-85de33e5366a admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:56.287373 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1394/2763] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:44:56 2022] POST /identity/v3/users => generated 285 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:44:56.291508 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf76d72-11c2-4c1d-8dc7-e77cae87da2e admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:56.301699 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf76d72-11c2-4c1d-8dc7-e77cae87da2e admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:56.306819 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ff6b20-6ae5-4a30-95f0-0106376b5dc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:56.307405 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ff6b20-6ae5-4a30-95f0-0106376b5dc3 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:56.307683 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ff6b20-6ae5-4a30-95f0-0106376b5dc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:56.307957 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ff6b20-6ae5-4a30-95f0-0106376b5dc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:56.308609 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ff6b20-6ae5-4a30-95f0-0106376b5dc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:56.310223 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ff6b20-6ae5-4a30-95f0-0106376b5dc3 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:56.320445 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1370/2764] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:44:56 2022] GET /identity/v3/roles => generated 1427 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:44:56.324162 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc95ad13-05fc-4189-8777-85de33e5366a admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:56.333788 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc95ad13-05fc-4189-8777-85de33e5366a admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:56.338273 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fdeefe-6195-4586-962b-9b50d9413ce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:56.338847 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fdeefe-6195-4586-962b-9b50d9413ce2 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:56.339125 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fdeefe-6195-4586-962b-9b50d9413ce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:56.339394 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fdeefe-6195-4586-962b-9b50d9413ce2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:56.340019 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fdeefe-6195-4586-962b-9b50d9413ce2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:56.341620 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fdeefe-6195-4586-962b-9b50d9413ce2 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:56.352700 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1395/2765] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:44:56 2022] GET /identity/v3/roles => generated 1427 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 17:44:56.358542 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ff6b20-6ae5-4a30-95f0-0106376b5dc3 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:56.369513 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ff6b20-6ae5-4a30-95f0-0106376b5dc3 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:56.374088 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c665741-be0f-4fb4-b64f-8c0832a4cdd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:56.374721 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c665741-be0f-4fb4-b64f-8c0832a4cdd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:56.374996 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c665741-be0f-4fb4-b64f-8c0832a4cdd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:56.375544 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c665741-be0f-4fb4-b64f-8c0832a4cdd1 admin admin] PATH_INFO: `/v3/projects/7e16c40def83414fba7906626516c259/users/d938ffb58f7d495da9a2fcfe3e2b336d/roles/b75cb360925e4ba5a65f14965c103ffd` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:56.391010 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c665741-be0f-4fb4-b64f-8c0832a4cdd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e16c40def83414fba7906626516c259, user_id=d938ffb58f7d495da9a2fcfe3e2b336d, role_id=b75cb360925e4ba5a65f14965c103ffd)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:56.394001 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c665741-be0f-4fb4-b64f-8c0832a4cdd1 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:56.406531 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1371/2766] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:44:56 2022] PUT /identity/v3/projects/7e16c40def83414fba7906626516c259/users/d938ffb58f7d495da9a2fcfe3e2b336d/roles/b75cb360925e4ba5a65f14965c103ffd => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:44:56.416918 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4359c59-8ad2-4d2e-80b1-f7eb3bdcaa59 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:56.417168 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4359c59-8ad2-4d2e-80b1-f7eb3bdcaa59 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:56.417486 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4359c59-8ad2-4d2e-80b1-f7eb3bdcaa59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:56.444774 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-c4359c59-8ad2-4d2e-80b1-f7eb3bdcaa59 None None] MFA Rules not processed for user `d938ffb58f7d495da9a2fcfe3e2b336d`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:56.446019 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-c4359c59-8ad2-4d2e-80b1-f7eb3bdcaa59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:56.511189 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1396/2767] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:44:56 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:44:56.522449 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c665741-be0f-4fb4-b64f-8c0832a4cdd1 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:56.524164 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-9c665741-be0f-4fb4-b64f-8c0832a4cdd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:56.546870 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:56.603529 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c665741-be0f-4fb4-b64f-8c0832a4cdd1 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:56.607331 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0e69b9-1e31-4562-91eb-2b9e4add9b7c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '039683b8d1d146c6add5ffa235d06f91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:56.607937 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0e69b9-1e31-4562-91eb-2b9e4add9b7c service glance] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:56.608224 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0e69b9-1e31-4562-91eb-2b9e4add9b7c service glance] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:56.608495 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0e69b9-1e31-4562-91eb-2b9e4add9b7c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:56.617486 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0e69b9-1e31-4562-91eb-2b9e4add9b7c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:56.618944 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0e69b9-1e31-4562-91eb-2b9e4add9b7c service glance] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:56.627882 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1372/2768] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:44:56 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:45:06.507778 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=84,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:06.623668 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=27 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:11.663025 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4359c59-8ad2-4d2e-80b1-f7eb3bdcaa59 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:11.664931 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-c4359c59-8ad2-4d2e-80b1-f7eb3bdcaa59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:11.733771 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4359c59-8ad2-4d2e-80b1-f7eb3bdcaa59 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:11.738764 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d57cef-d35e-465a-9ce3-46ee2a0704dd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '134ef6039cef468bae0e3f3be79b8073', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:11.739366 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d57cef-d35e-465a-9ce3-46ee2a0704dd service placement] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:11.739669 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d57cef-d35e-465a-9ce3-46ee2a0704dd service placement] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:11.739941 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d57cef-d35e-465a-9ce3-46ee2a0704dd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:11.741909 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-62d57cef-d35e-465a-9ce3-46ee2a0704dd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:11.753654 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d57cef-d35e-465a-9ce3-46ee2a0704dd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:11.755239 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d57cef-d35e-465a-9ce3-46ee2a0704dd service placement] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:11.765248 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1397/2769] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:45:11 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:45:21.760895 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=32 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:42.528866 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0e69b9-1e31-4562-91eb-2b9e4add9b7c service glance] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:42.590570 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0e69b9-1e31-4562-91eb-2b9e4add9b7c service glance] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:42.595113 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3476a3b-a9ee-49d3-ac95-3f2c16dc7b99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:42.595723 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3476a3b-a9ee-49d3-ac95-3f2c16dc7b99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:42.596005 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3476a3b-a9ee-49d3-ac95-3f2c16dc7b99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:42.596277 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3476a3b-a9ee-49d3-ac95-3f2c16dc7b99 admin admin] PATH_INFO: `/v3/users/d938ffb58f7d495da9a2fcfe3e2b336d` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:42.597415 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3476a3b-a9ee-49d3-ac95-3f2c16dc7b99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d938ffb58f7d495da9a2fcfe3e2b336d)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:42.599335 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3476a3b-a9ee-49d3-ac95-3f2c16dc7b99 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:42.643595 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-e3476a3b-a9ee-49d3-ac95-3f2c16dc7b99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd938ffb58f7d495da9a2fcfe3e2b336d'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:42.644234 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-e3476a3b-a9ee-49d3-ac95-3f2c16dc7b99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd938ffb58f7d495da9a2fcfe3e2b336d'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:42.656324 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-e3476a3b-a9ee-49d3-ac95-3f2c16dc7b99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd938ffb58f7d495da9a2fcfe3e2b336d'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:42.663471 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-e3476a3b-a9ee-49d3-ac95-3f2c16dc7b99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd938ffb58f7d495da9a2fcfe3e2b336d'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:42.664222 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-e3476a3b-a9ee-49d3-ac95-3f2c16dc7b99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd938ffb58f7d495da9a2fcfe3e2b336d'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:42.677102 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1373/2770] 10.222.0.102 () {58 vars in 1229 bytes} [Thu Sep 1 17:45:42 2022] DELETE /identity/v3/users/d938ffb58f7d495da9a2fcfe3e2b336d => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:45:42.681656 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d48c78-bfb0-48f0-b17c-dffeb17584bc None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:42.682145 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d48c78-bfb0-48f0-b17c-dffeb17584bc None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:42.682257 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d48c78-bfb0-48f0-b17c-dffeb17584bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:42.700486 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-32d48c78-bfb0-48f0-b17c-dffeb17584bc None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:45:42.701843 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-32d48c78-bfb0-48f0-b17c-dffeb17584bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:42.758767 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1398/2771] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:45:42 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:45:42.768708 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3476a3b-a9ee-49d3-ac95-3f2c16dc7b99 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:42.770156 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-e3476a3b-a9ee-49d3-ac95-3f2c16dc7b99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:42.826226 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3476a3b-a9ee-49d3-ac95-3f2c16dc7b99 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:42.830023 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc310330-c801-4290-9a18-40a591eff717 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:42.830642 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc310330-c801-4290-9a18-40a591eff717 service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:42.830900 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc310330-c801-4290-9a18-40a591eff717 service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:42.831170 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc310330-c801-4290-9a18-40a591eff717 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:42.840597 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc310330-c801-4290-9a18-40a591eff717 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:42.842028 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc310330-c801-4290-9a18-40a591eff717 service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:42.851378 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1374/2772] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:45:42 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:45:43.115316 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d48c78-bfb0-48f0-b17c-dffeb17584bc None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:43.116734 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-32d48c78-bfb0-48f0-b17c-dffeb17584bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:43.128979 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d48c78-bfb0-48f0-b17c-dffeb17584bc None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:43.133618 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fae4960-66b3-4108-a6e4-c33315625e3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:43.134244 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fae4960-66b3-4108-a6e4-c33315625e3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:43.134518 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fae4960-66b3-4108-a6e4-c33315625e3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:43.134787 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fae4960-66b3-4108-a6e4-c33315625e3f admin admin] PATH_INFO: `/v3/projects/7e16c40def83414fba7906626516c259` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:43.135884 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fae4960-66b3-4108-a6e4-c33315625e3f admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e16c40def83414fba7906626516c259)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:43.137804 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fae4960-66b3-4108-a6e4-c33315625e3f admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:43.185599 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9fae4960-66b3-4108-a6e4-c33315625e3f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e16c40def83414fba7906626516c259', 'request_id': 'req-9fae4960-66b3-4108-a6e4-c33315625e3f'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:43.193236 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9fae4960-66b3-4108-a6e4-c33315625e3f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e16c40def83414fba7906626516c259', 'request_id': 'req-9fae4960-66b3-4108-a6e4-c33315625e3f'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:43.200420 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9fae4960-66b3-4108-a6e4-c33315625e3f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e16c40def83414fba7906626516c259', 'request_id': 'req-9fae4960-66b3-4108-a6e4-c33315625e3f'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:43.210845 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9fae4960-66b3-4108-a6e4-c33315625e3f admin admin] The token cache is being invalidate because project 7e16c40def83414fba7906626516c259 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:45:43.211136 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9fae4960-66b3-4108-a6e4-c33315625e3f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:43.211758 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9fae4960-66b3-4108-a6e4-c33315625e3f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:43.213218 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1399/2773] 10.222.0.102 () {58 vars in 1238 bytes} [Thu Sep 1 17:45:43 2022] DELETE /identity/v3/projects/7e16c40def83414fba7906626516c259 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:45:43.218553 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4433ffb-5268-4caf-8b68-2ac0d58d6d0e None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:43.218841 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4433ffb-5268-4caf-8b68-2ac0d58d6d0e None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:43.219118 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4433ffb-5268-4caf-8b68-2ac0d58d6d0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:43.238975 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-b4433ffb-5268-4caf-8b68-2ac0d58d6d0e None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:45:43.240163 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-b4433ffb-5268-4caf-8b68-2ac0d58d6d0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:43.256381 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:43.300130 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1375/2774] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:45:43 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:45:43.307917 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a70c201-c788-4807-ba51-07ddefc01491 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:43.308202 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a70c201-c788-4807-ba51-07ddefc01491 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:43.308476 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a70c201-c788-4807-ba51-07ddefc01491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:43.327558 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-2a70c201-c788-4807-ba51-07ddefc01491 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:45:43.328732 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-2a70c201-c788-4807-ba51-07ddefc01491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:43.336358 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1400/2775] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:45:43 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:45:43.340891 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4433ffb-5268-4caf-8b68-2ac0d58d6d0e None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:43.350390 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4433ffb-5268-4caf-8b68-2ac0d58d6d0e None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:43.354883 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecbfe88-42fe-43c5-b396-eb00b0584cee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:43.356333 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecbfe88-42fe-43c5-b396-eb00b0584cee admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:43.356618 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecbfe88-42fe-43c5-b396-eb00b0584cee admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:43.356891 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecbfe88-42fe-43c5-b396-eb00b0584cee admin admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:43.357617 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecbfe88-42fe-43c5-b396-eb00b0584cee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:43.357967 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecbfe88-42fe-43c5-b396-eb00b0584cee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:43.359554 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecbfe88-42fe-43c5-b396-eb00b0584cee admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:43.370931 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1376/2776] 10.222.0.102 () {58 vars in 1158 bytes} [Thu Sep 1 17:45:43 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:43.375223 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a70c201-c788-4807-ba51-07ddefc01491 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:43.384253 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a70c201-c788-4807-ba51-07ddefc01491 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:43.388857 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7415c2e0-7e92-42f1-a414-8cce4d0b991c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:43.389541 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7415c2e0-7e92-42f1-a414-8cce4d0b991c admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:43.389818 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7415c2e0-7e92-42f1-a414-8cce4d0b991c admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:43.390085 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7415c2e0-7e92-42f1-a414-8cce4d0b991c admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:43.390715 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7415c2e0-7e92-42f1-a414-8cce4d0b991c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:43.392595 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7415c2e0-7e92-42f1-a414-8cce4d0b991c admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:43.419475 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1401/2777] 10.222.0.102 () {60 vars in 1158 bytes} [Thu Sep 1 17:45:43 2022] POST /identity/v3/projects => generated 386 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:45:43.423582 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecbfe88-42fe-43c5-b396-eb00b0584cee admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:43.433257 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecbfe88-42fe-43c5-b396-eb00b0584cee admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:43.490065 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96afed5b-ca96-4e95-a2e8-8d2d8a89f0a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:43.490669 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96afed5b-ca96-4e95-a2e8-8d2d8a89f0a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:43.490953 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96afed5b-ca96-4e95-a2e8-8d2d8a89f0a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:43.491229 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96afed5b-ca96-4e95-a2e8-8d2d8a89f0a7 admin admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:43.491859 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96afed5b-ca96-4e95-a2e8-8d2d8a89f0a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:43.493807 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96afed5b-ca96-4e95-a2e8-8d2d8a89f0a7 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:43.514983 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1377/2778] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:45:43 2022] POST /identity/v3/users => generated 294 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:45:43.519554 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7415c2e0-7e92-42f1-a414-8cce4d0b991c admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:43.530599 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7415c2e0-7e92-42f1-a414-8cce4d0b991c admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:43.535177 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344b5afa-feb4-44fb-b293-e85cefc4367d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:43.535766 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344b5afa-feb4-44fb-b293-e85cefc4367d admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:43.536046 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344b5afa-feb4-44fb-b293-e85cefc4367d admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:43.536319 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344b5afa-feb4-44fb-b293-e85cefc4367d admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:43.536959 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344b5afa-feb4-44fb-b293-e85cefc4367d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:43.538563 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344b5afa-feb4-44fb-b293-e85cefc4367d admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:43.548305 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1402/2779] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:45:43 2022] GET /identity/v3/roles => generated 1427 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:45:43.552450 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96afed5b-ca96-4e95-a2e8-8d2d8a89f0a7 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:43.562354 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96afed5b-ca96-4e95-a2e8-8d2d8a89f0a7 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:43.566979 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a38b769-5b68-4012-aeae-23dd51b525d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:43.567564 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a38b769-5b68-4012-aeae-23dd51b525d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:43.567849 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a38b769-5b68-4012-aeae-23dd51b525d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:43.568127 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a38b769-5b68-4012-aeae-23dd51b525d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:43.568764 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a38b769-5b68-4012-aeae-23dd51b525d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:43.570413 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a38b769-5b68-4012-aeae-23dd51b525d9 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:43.581298 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1378/2780] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:45:43 2022] GET /identity/v3/roles => generated 1427 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:45:43.585172 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344b5afa-feb4-44fb-b293-e85cefc4367d admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:43.595528 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344b5afa-feb4-44fb-b293-e85cefc4367d admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:43.600032 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e91a1f-de9f-45cf-901f-eef59f548c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:43.600665 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e91a1f-de9f-45cf-901f-eef59f548c6a admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:43.600948 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e91a1f-de9f-45cf-901f-eef59f548c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:43.601224 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e91a1f-de9f-45cf-901f-eef59f548c6a admin admin] PATH_INFO: `/v3/projects/e3a5ea46c0ba4b10acf67da7e207d490/users/33fe2193f1de4250a165a8ce0f07dffa/roles/b75cb360925e4ba5a65f14965c103ffd` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:43.616351 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e91a1f-de9f-45cf-901f-eef59f548c6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3a5ea46c0ba4b10acf67da7e207d490, user_id=33fe2193f1de4250a165a8ce0f07dffa, role_id=b75cb360925e4ba5a65f14965c103ffd)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:43.619015 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e91a1f-de9f-45cf-901f-eef59f548c6a admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:43.631858 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1403/2781] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:45:43 2022] PUT /identity/v3/projects/e3a5ea46c0ba4b10acf67da7e207d490/users/33fe2193f1de4250a165a8ce0f07dffa/roles/b75cb360925e4ba5a65f14965c103ffd => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:45:43.695489 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794dd6b7-fa90-46a8-b58f-2dd7df086163 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:43.695779 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794dd6b7-fa90-46a8-b58f-2dd7df086163 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:43.696054 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794dd6b7-fa90-46a8-b58f-2dd7df086163 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:43.722974 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-794dd6b7-fa90-46a8-b58f-2dd7df086163 None None] MFA Rules not processed for user `33fe2193f1de4250a165a8ce0f07dffa`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:45:43.724185 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-794dd6b7-fa90-46a8-b58f-2dd7df086163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:43.787753 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1379/2782] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:45:43 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:45:43.799398 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e91a1f-de9f-45cf-901f-eef59f548c6a admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:43.801125 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-d5e91a1f-de9f-45cf-901f-eef59f548c6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:43.823894 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:43.861056 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e91a1f-de9f-45cf-901f-eef59f548c6a admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:43.864722 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f26ca77-00f5-463f-b7a3-cd57a21cbd0e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '523ed589872a45a09b6a7675a7fb0b75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:43.865348 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f26ca77-00f5-463f-b7a3-cd57a21cbd0e service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:43.865628 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f26ca77-00f5-463f-b7a3-cd57a21cbd0e service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:43.865899 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f26ca77-00f5-463f-b7a3-cd57a21cbd0e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:43.874573 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f26ca77-00f5-463f-b7a3-cd57a21cbd0e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:43.875979 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f26ca77-00f5-463f-b7a3-cd57a21cbd0e service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:43.884608 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1404/2783] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:45:43 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:45:46.428666 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794dd6b7-fa90-46a8-b58f-2dd7df086163 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:46.447648 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:46.492832 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794dd6b7-fa90-46a8-b58f-2dd7df086163 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:46.497291 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcf423d-de3e-4c86-a5ed-353bdc7cca64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:46.497928 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcf423d-de3e-4c86-a5ed-353bdc7cca64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:46.498204 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcf423d-de3e-4c86-a5ed-353bdc7cca64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:46.498469 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcf423d-de3e-4c86-a5ed-353bdc7cca64 admin admin] PATH_INFO: `/v3/users/33fe2193f1de4250a165a8ce0f07dffa` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:46.499559 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcf423d-de3e-4c86-a5ed-353bdc7cca64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=33fe2193f1de4250a165a8ce0f07dffa)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:46.501480 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcf423d-de3e-4c86-a5ed-353bdc7cca64 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:46.547105 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-4dcf423d-de3e-4c86-a5ed-353bdc7cca64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33fe2193f1de4250a165a8ce0f07dffa'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:46.547866 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-4dcf423d-de3e-4c86-a5ed-353bdc7cca64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33fe2193f1de4250a165a8ce0f07dffa'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:46.559804 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-4dcf423d-de3e-4c86-a5ed-353bdc7cca64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33fe2193f1de4250a165a8ce0f07dffa'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:46.566347 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-4dcf423d-de3e-4c86-a5ed-353bdc7cca64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33fe2193f1de4250a165a8ce0f07dffa'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:46.567008 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-4dcf423d-de3e-4c86-a5ed-353bdc7cca64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33fe2193f1de4250a165a8ce0f07dffa'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:46.579512 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1380/2784] 10.222.0.102 () {58 vars in 1229 bytes} [Thu Sep 1 17:45:46 2022] DELETE /identity/v3/users/33fe2193f1de4250a165a8ce0f07dffa => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:45:46.584195 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f7fa22-119c-47e1-8e8e-2a46fb05b4e0 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:46.584656 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f7fa22-119c-47e1-8e8e-2a46fb05b4e0 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:46.584773 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f7fa22-119c-47e1-8e8e-2a46fb05b4e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:46.604033 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-f1f7fa22-119c-47e1-8e8e-2a46fb05b4e0 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:45:46.605755 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-f1f7fa22-119c-47e1-8e8e-2a46fb05b4e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:46.663955 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1405/2785] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:45:46 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:45:46.673974 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcf423d-de3e-4c86-a5ed-353bdc7cca64 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:46.675420 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-4dcf423d-de3e-4c86-a5ed-353bdc7cca64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:46.735414 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcf423d-de3e-4c86-a5ed-353bdc7cca64 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:46.739035 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad73598-7a1b-4811-8de3-6239aad8d051 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:46.739639 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad73598-7a1b-4811-8de3-6239aad8d051 service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:46.739910 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad73598-7a1b-4811-8de3-6239aad8d051 service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:46.740177 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad73598-7a1b-4811-8de3-6239aad8d051 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:46.749512 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad73598-7a1b-4811-8de3-6239aad8d051 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:46.750920 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad73598-7a1b-4811-8de3-6239aad8d051 service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:46.760065 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1381/2786] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:45:46 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:45:47.019515 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f7fa22-119c-47e1-8e8e-2a46fb05b4e0 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:47.021077 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-f1f7fa22-119c-47e1-8e8e-2a46fb05b4e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:47.036278 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f7fa22-119c-47e1-8e8e-2a46fb05b4e0 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:47.048815 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c02a33-ecf9-46a7-9339-fd7cea01b82f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:47.049501 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c02a33-ecf9-46a7-9339-fd7cea01b82f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:47.049786 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c02a33-ecf9-46a7-9339-fd7cea01b82f admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:47.050063 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c02a33-ecf9-46a7-9339-fd7cea01b82f admin admin] PATH_INFO: `/v3/projects/e3a5ea46c0ba4b10acf67da7e207d490` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:47.051262 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c02a33-ecf9-46a7-9339-fd7cea01b82f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3a5ea46c0ba4b10acf67da7e207d490)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:47.053176 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c02a33-ecf9-46a7-9339-fd7cea01b82f admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:47.100135 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b5c02a33-ecf9-46a7-9339-fd7cea01b82f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3a5ea46c0ba4b10acf67da7e207d490', 'request_id': 'req-b5c02a33-ecf9-46a7-9339-fd7cea01b82f'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:47.107649 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b5c02a33-ecf9-46a7-9339-fd7cea01b82f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3a5ea46c0ba4b10acf67da7e207d490', 'request_id': 'req-b5c02a33-ecf9-46a7-9339-fd7cea01b82f'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:47.113984 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b5c02a33-ecf9-46a7-9339-fd7cea01b82f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3a5ea46c0ba4b10acf67da7e207d490', 'request_id': 'req-b5c02a33-ecf9-46a7-9339-fd7cea01b82f'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:47.124216 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b5c02a33-ecf9-46a7-9339-fd7cea01b82f admin admin] The token cache is being invalidate because project e3a5ea46c0ba4b10acf67da7e207d490 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:45:47.124509 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b5c02a33-ecf9-46a7-9339-fd7cea01b82f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:47.125134 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b5c02a33-ecf9-46a7-9339-fd7cea01b82f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:47.126668 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1406/2787] 10.222.0.102 () {58 vars in 1238 bytes} [Thu Sep 1 17:45:47 2022] DELETE /identity/v3/projects/e3a5ea46c0ba4b10acf67da7e207d490 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:45:47.131485 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c588f3-da7d-4a3c-9290-a73cd0024daf None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:47.131770 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c588f3-da7d-4a3c-9290-a73cd0024daf None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:47.132041 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c588f3-da7d-4a3c-9290-a73cd0024daf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:47.151237 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-d3c588f3-da7d-4a3c-9290-a73cd0024daf None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:45:47.152986 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-d3c588f3-da7d-4a3c-9290-a73cd0024daf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:47.187156 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:47.213149 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1382/2788] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:45:47 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:45:47.221166 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbac1aad-055b-4e72-8a17-fb7ea484e7b5 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:47.221494 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbac1aad-055b-4e72-8a17-fb7ea484e7b5 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:47.221770 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbac1aad-055b-4e72-8a17-fb7ea484e7b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:47.242523 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-dbac1aad-055b-4e72-8a17-fb7ea484e7b5 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:45:47.243718 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-dbac1aad-055b-4e72-8a17-fb7ea484e7b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:47.251292 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1407/2789] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:45:47 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:45:47.255283 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c588f3-da7d-4a3c-9290-a73cd0024daf None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:47.265526 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c588f3-da7d-4a3c-9290-a73cd0024daf None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:47.270070 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929f3384-f81d-4698-8fc4-556bf41f3adb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:47.270657 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929f3384-f81d-4698-8fc4-556bf41f3adb admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:47.270937 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929f3384-f81d-4698-8fc4-556bf41f3adb admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:47.271209 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929f3384-f81d-4698-8fc4-556bf41f3adb admin admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:47.271845 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929f3384-f81d-4698-8fc4-556bf41f3adb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:47.272178 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929f3384-f81d-4698-8fc4-556bf41f3adb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:47.273796 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929f3384-f81d-4698-8fc4-556bf41f3adb admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:47.285466 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1383/2790] 10.222.0.102 () {58 vars in 1158 bytes} [Thu Sep 1 17:45:47 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:47.289432 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbac1aad-055b-4e72-8a17-fb7ea484e7b5 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:47.298722 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbac1aad-055b-4e72-8a17-fb7ea484e7b5 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:47.303307 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa21b58-21b3-4405-8e70-e0bb67f8a185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:47.303895 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa21b58-21b3-4405-8e70-e0bb67f8a185 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:47.304174 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa21b58-21b3-4405-8e70-e0bb67f8a185 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:47.304446 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa21b58-21b3-4405-8e70-e0bb67f8a185 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:47.305086 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa21b58-21b3-4405-8e70-e0bb67f8a185 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:47.307020 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa21b58-21b3-4405-8e70-e0bb67f8a185 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:47.333249 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1408/2791] 10.222.0.102 () {60 vars in 1158 bytes} [Thu Sep 1 17:45:47 2022] POST /identity/v3/projects => generated 370 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:45:47.337563 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929f3384-f81d-4698-8fc4-556bf41f3adb admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:47.348221 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929f3384-f81d-4698-8fc4-556bf41f3adb admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:47.405171 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06112c99-bc74-4085-9f30-c9cb46874884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:47.405800 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06112c99-bc74-4085-9f30-c9cb46874884 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:47.406076 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06112c99-bc74-4085-9f30-c9cb46874884 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:47.406349 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06112c99-bc74-4085-9f30-c9cb46874884 admin admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:47.406981 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06112c99-bc74-4085-9f30-c9cb46874884 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:47.408891 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06112c99-bc74-4085-9f30-c9cb46874884 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:47.428876 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1384/2792] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:45:47 2022] POST /identity/v3/users => generated 286 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:45:47.433077 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa21b58-21b3-4405-8e70-e0bb67f8a185 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:47.443411 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa21b58-21b3-4405-8e70-e0bb67f8a185 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:47.447842 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70f2cbc-2051-489a-b929-2c3afa0584d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:47.448426 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70f2cbc-2051-489a-b929-2c3afa0584d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:47.448704 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70f2cbc-2051-489a-b929-2c3afa0584d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:47.449480 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70f2cbc-2051-489a-b929-2c3afa0584d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:47.450114 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70f2cbc-2051-489a-b929-2c3afa0584d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:47.451676 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70f2cbc-2051-489a-b929-2c3afa0584d9 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:47.461601 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1409/2793] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:45:47 2022] GET /identity/v3/roles => generated 1427 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:45:47.466020 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06112c99-bc74-4085-9f30-c9cb46874884 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:47.475745 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06112c99-bc74-4085-9f30-c9cb46874884 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:47.480344 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a59030-466e-4903-b513-e93dd92c4179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:47.480924 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a59030-466e-4903-b513-e93dd92c4179 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:47.481506 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a59030-466e-4903-b513-e93dd92c4179 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:47.481783 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a59030-466e-4903-b513-e93dd92c4179 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:47.482413 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a59030-466e-4903-b513-e93dd92c4179 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:47.483982 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a59030-466e-4903-b513-e93dd92c4179 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:47.494318 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1385/2794] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:45:47 2022] GET /identity/v3/roles => generated 1427 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:45:47.498159 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70f2cbc-2051-489a-b929-2c3afa0584d9 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:47.508421 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70f2cbc-2051-489a-b929-2c3afa0584d9 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:47.513711 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5720e5-264e-41bd-84f9-c9ad3609911d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:47.514343 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5720e5-264e-41bd-84f9-c9ad3609911d admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:47.514611 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5720e5-264e-41bd-84f9-c9ad3609911d admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:47.514878 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5720e5-264e-41bd-84f9-c9ad3609911d admin admin] PATH_INFO: `/v3/projects/a86b5dcf986245be98e347ae4312c9fe/users/91196976eaae4314a0f57d3f2407dd1a/roles/b75cb360925e4ba5a65f14965c103ffd` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:47.517047 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:47.531993 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5720e5-264e-41bd-84f9-c9ad3609911d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a86b5dcf986245be98e347ae4312c9fe, user_id=91196976eaae4314a0f57d3f2407dd1a, role_id=b75cb360925e4ba5a65f14965c103ffd)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:47.534651 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5720e5-264e-41bd-84f9-c9ad3609911d admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:47.546858 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1410/2795] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:45:47 2022] PUT /identity/v3/projects/a86b5dcf986245be98e347ae4312c9fe/users/91196976eaae4314a0f57d3f2407dd1a/roles/b75cb360925e4ba5a65f14965c103ffd => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:45:47.558004 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352ca316-97f0-4b80-b186-3b0a2b218ab5 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:47.558290 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352ca316-97f0-4b80-b186-3b0a2b218ab5 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:47.558560 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352ca316-97f0-4b80-b186-3b0a2b218ab5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:47.585711 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-352ca316-97f0-4b80-b186-3b0a2b218ab5 None None] MFA Rules not processed for user `91196976eaae4314a0f57d3f2407dd1a`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:45:47.586898 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-352ca316-97f0-4b80-b186-3b0a2b218ab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:47.649651 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1386/2796] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:45:47 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:45:47.661407 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5720e5-264e-41bd-84f9-c9ad3609911d admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:47.663144 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-9a5720e5-264e-41bd-84f9-c9ad3609911d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:47.719499 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5720e5-264e-41bd-84f9-c9ad3609911d admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:47.723057 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f4c8fe-12bb-4d09-b0eb-bd25a355547a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '039683b8d1d146c6add5ffa235d06f91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:47.723659 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f4c8fe-12bb-4d09-b0eb-bd25a355547a service glance] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:47.723938 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f4c8fe-12bb-4d09-b0eb-bd25a355547a service glance] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:47.724210 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f4c8fe-12bb-4d09-b0eb-bd25a355547a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:47.733264 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f4c8fe-12bb-4d09-b0eb-bd25a355547a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:47.734715 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f4c8fe-12bb-4d09-b0eb-bd25a355547a service glance] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:47.743490 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1411/2797] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:45:47 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:45:57.646768 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=81,keystone:INSERT=3 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:57.739390 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=33,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:23.818374 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352ca316-97f0-4b80-b186-3b0a2b218ab5 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:23.880719 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352ca316-97f0-4b80-b186-3b0a2b218ab5 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:23.885546 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb5fe2d-648d-414a-92a2-1cd8ece25811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:23.886122 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb5fe2d-648d-414a-92a2-1cd8ece25811 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:23.886403 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb5fe2d-648d-414a-92a2-1cd8ece25811 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:23.886676 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb5fe2d-648d-414a-92a2-1cd8ece25811 admin admin] PATH_INFO: `/v3/users/91196976eaae4314a0f57d3f2407dd1a` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:23.887796 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb5fe2d-648d-414a-92a2-1cd8ece25811 admin admin] RBAC: Authorizing `identity:delete_user(user_id=91196976eaae4314a0f57d3f2407dd1a)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:23.889797 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb5fe2d-648d-414a-92a2-1cd8ece25811 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:23.937924 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-fcb5fe2d-648d-414a-92a2-1cd8ece25811 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91196976eaae4314a0f57d3f2407dd1a'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:46:23.938586 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-fcb5fe2d-648d-414a-92a2-1cd8ece25811 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91196976eaae4314a0f57d3f2407dd1a'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:46:23.951165 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-fcb5fe2d-648d-414a-92a2-1cd8ece25811 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91196976eaae4314a0f57d3f2407dd1a'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:46:23.959407 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-fcb5fe2d-648d-414a-92a2-1cd8ece25811 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91196976eaae4314a0f57d3f2407dd1a'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:46:23.960069 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-fcb5fe2d-648d-414a-92a2-1cd8ece25811 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91196976eaae4314a0f57d3f2407dd1a'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:46:23.972987 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1387/2798] 10.222.0.102 () {58 vars in 1229 bytes} [Thu Sep 1 17:46:23 2022] DELETE /identity/v3/users/91196976eaae4314a0f57d3f2407dd1a => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:46:23.977407 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957170f5-1c32-49cb-a98f-564e6a30f891 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:23.977881 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957170f5-1c32-49cb-a98f-564e6a30f891 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:23.977988 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957170f5-1c32-49cb-a98f-564e6a30f891 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:23.996560 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-957170f5-1c32-49cb-a98f-564e6a30f891 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:46:23.997763 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-957170f5-1c32-49cb-a98f-564e6a30f891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:46:24.054382 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1412/2799] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:46:23 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:46:24.064638 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb5fe2d-648d-414a-92a2-1cd8ece25811 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:24.066088 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-fcb5fe2d-648d-414a-92a2-1cd8ece25811 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:46:24.124153 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb5fe2d-648d-414a-92a2-1cd8ece25811 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:24.127839 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fe6d5d-347f-43a9-8499-304bef746f03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:24.128439 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fe6d5d-347f-43a9-8499-304bef746f03 service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:24.128717 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fe6d5d-347f-43a9-8499-304bef746f03 service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:24.128988 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fe6d5d-347f-43a9-8499-304bef746f03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:24.138436 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fe6d5d-347f-43a9-8499-304bef746f03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:24.139992 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fe6d5d-347f-43a9-8499-304bef746f03 service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:24.149615 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1388/2800] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:46:24 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 17:46:24.414360 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957170f5-1c32-49cb-a98f-564e6a30f891 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:24.415904 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-957170f5-1c32-49cb-a98f-564e6a30f891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:46:24.427673 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957170f5-1c32-49cb-a98f-564e6a30f891 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:24.432510 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b925bd04-c781-4302-b4f3-8d5eb2287101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:24.433141 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b925bd04-c781-4302-b4f3-8d5eb2287101 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:24.433468 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b925bd04-c781-4302-b4f3-8d5eb2287101 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:24.433738 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b925bd04-c781-4302-b4f3-8d5eb2287101 admin admin] PATH_INFO: `/v3/projects/a86b5dcf986245be98e347ae4312c9fe` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:24.434900 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b925bd04-c781-4302-b4f3-8d5eb2287101 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a86b5dcf986245be98e347ae4312c9fe)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:24.437379 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b925bd04-c781-4302-b4f3-8d5eb2287101 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:24.484879 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b925bd04-c781-4302-b4f3-8d5eb2287101 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a86b5dcf986245be98e347ae4312c9fe', 'request_id': 'req-b925bd04-c781-4302-b4f3-8d5eb2287101'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:46:24.492293 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b925bd04-c781-4302-b4f3-8d5eb2287101 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a86b5dcf986245be98e347ae4312c9fe', 'request_id': 'req-b925bd04-c781-4302-b4f3-8d5eb2287101'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:46:24.499000 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b925bd04-c781-4302-b4f3-8d5eb2287101 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a86b5dcf986245be98e347ae4312c9fe', 'request_id': 'req-b925bd04-c781-4302-b4f3-8d5eb2287101'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:46:24.509232 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b925bd04-c781-4302-b4f3-8d5eb2287101 admin admin] The token cache is being invalidate because project a86b5dcf986245be98e347ae4312c9fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:46:24.509548 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b925bd04-c781-4302-b4f3-8d5eb2287101 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:46:24.510180 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b925bd04-c781-4302-b4f3-8d5eb2287101 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:46:24.511656 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1413/2801] 10.222.0.102 () {58 vars in 1238 bytes} [Thu Sep 1 17:46:24 2022] DELETE /identity/v3/projects/a86b5dcf986245be98e347ae4312c9fe => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:46:24.517207 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d741e71-de7b-4f51-92bb-13852b67409a None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:24.517527 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d741e71-de7b-4f51-92bb-13852b67409a None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:24.517804 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d741e71-de7b-4f51-92bb-13852b67409a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:24.538154 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-4d741e71-de7b-4f51-92bb-13852b67409a None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:46:24.539358 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-4d741e71-de7b-4f51-92bb-13852b67409a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:46:24.556367 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:24.599120 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1389/2802] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:46:24 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:46:24.607021 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c508298c-4fcb-41a8-a09c-1cbed5121750 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:24.607309 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c508298c-4fcb-41a8-a09c-1cbed5121750 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:24.607645 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c508298c-4fcb-41a8-a09c-1cbed5121750 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:24.627636 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-c508298c-4fcb-41a8-a09c-1cbed5121750 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:46:24.628824 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-c508298c-4fcb-41a8-a09c-1cbed5121750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:46:24.636595 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1414/2803] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:46:24 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:46:24.641159 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d741e71-de7b-4f51-92bb-13852b67409a None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:24.651649 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d741e71-de7b-4f51-92bb-13852b67409a None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:24.656125 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e8eba7-76e1-44d0-bd2a-61fab19d5b5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:24.656703 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e8eba7-76e1-44d0-bd2a-61fab19d5b5d admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:24.656978 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e8eba7-76e1-44d0-bd2a-61fab19d5b5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:24.657248 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e8eba7-76e1-44d0-bd2a-61fab19d5b5d admin admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:24.657909 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e8eba7-76e1-44d0-bd2a-61fab19d5b5d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:46:24.658209 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e8eba7-76e1-44d0-bd2a-61fab19d5b5d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:24.659771 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e8eba7-76e1-44d0-bd2a-61fab19d5b5d admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:24.670889 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1390/2804] 10.222.0.102 () {58 vars in 1158 bytes} [Thu Sep 1 17:46:24 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:46:24.675036 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c508298c-4fcb-41a8-a09c-1cbed5121750 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:24.684079 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c508298c-4fcb-41a8-a09c-1cbed5121750 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:24.689176 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d10ef95-7255-4e6d-bc8b-20731f19c0bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:24.689790 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d10ef95-7255-4e6d-bc8b-20731f19c0bc admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:24.690070 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d10ef95-7255-4e6d-bc8b-20731f19c0bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:24.690342 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d10ef95-7255-4e6d-bc8b-20731f19c0bc admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:24.690962 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d10ef95-7255-4e6d-bc8b-20731f19c0bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:24.692869 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d10ef95-7255-4e6d-bc8b-20731f19c0bc admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:24.719511 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1415/2805] 10.222.0.102 () {60 vars in 1158 bytes} [Thu Sep 1 17:46:24 2022] POST /identity/v3/projects => generated 372 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:46:24.723690 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e8eba7-76e1-44d0-bd2a-61fab19d5b5d admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:24.734302 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e8eba7-76e1-44d0-bd2a-61fab19d5b5d admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:24.788984 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7253153e-08c7-496b-a3f2-d58f740a72a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:24.789625 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7253153e-08c7-496b-a3f2-d58f740a72a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:24.789921 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7253153e-08c7-496b-a3f2-d58f740a72a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:24.790187 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7253153e-08c7-496b-a3f2-d58f740a72a9 admin admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:24.790798 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7253153e-08c7-496b-a3f2-d58f740a72a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:24.792682 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7253153e-08c7-496b-a3f2-d58f740a72a9 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:24.815083 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1391/2806] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:46:24 2022] POST /identity/v3/users => generated 287 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:46:24.819226 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d10ef95-7255-4e6d-bc8b-20731f19c0bc admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:24.829246 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d10ef95-7255-4e6d-bc8b-20731f19c0bc admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:24.833754 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29a122f-fcd0-4ea7-8280-429fdcb26bab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:24.834338 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29a122f-fcd0-4ea7-8280-429fdcb26bab admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:24.834622 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29a122f-fcd0-4ea7-8280-429fdcb26bab admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:24.834892 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29a122f-fcd0-4ea7-8280-429fdcb26bab admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:24.835520 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29a122f-fcd0-4ea7-8280-429fdcb26bab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:24.837276 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29a122f-fcd0-4ea7-8280-429fdcb26bab admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:24.847001 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1416/2807] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:46:24 2022] GET /identity/v3/roles => generated 1427 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:46:24.850961 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7253153e-08c7-496b-a3f2-d58f740a72a9 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:24.862151 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7253153e-08c7-496b-a3f2-d58f740a72a9 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:24.866721 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77357c7d-012e-4ef0-9dd6-6c8519fa84a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:24.867316 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77357c7d-012e-4ef0-9dd6-6c8519fa84a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:24.867596 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77357c7d-012e-4ef0-9dd6-6c8519fa84a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:24.867865 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77357c7d-012e-4ef0-9dd6-6c8519fa84a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:24.868528 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77357c7d-012e-4ef0-9dd6-6c8519fa84a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:24.870148 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77357c7d-012e-4ef0-9dd6-6c8519fa84a2 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:24.883107 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1392/2808] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:46:24 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:46:24.885709 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29a122f-fcd0-4ea7-8280-429fdcb26bab admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:24.896859 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29a122f-fcd0-4ea7-8280-429fdcb26bab admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:24.901692 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc7fde7-c9ea-4892-a106-f7d3fc6b8c03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:24.902345 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc7fde7-c9ea-4892-a106-f7d3fc6b8c03 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:24.902621 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc7fde7-c9ea-4892-a106-f7d3fc6b8c03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:24.902895 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc7fde7-c9ea-4892-a106-f7d3fc6b8c03 admin admin] PATH_INFO: `/v3/projects/85538f245b5b4f4d8799169e2857278d/users/cfa2b12905ac4709947cbeb9db264f33/roles/b75cb360925e4ba5a65f14965c103ffd` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:24.917236 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc7fde7-c9ea-4892-a106-f7d3fc6b8c03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=85538f245b5b4f4d8799169e2857278d, user_id=cfa2b12905ac4709947cbeb9db264f33, role_id=b75cb360925e4ba5a65f14965c103ffd)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:24.919891 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc7fde7-c9ea-4892-a106-f7d3fc6b8c03 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:24.933827 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1417/2809] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:46:24 2022] PUT /identity/v3/projects/85538f245b5b4f4d8799169e2857278d/users/cfa2b12905ac4709947cbeb9db264f33/roles/b75cb360925e4ba5a65f14965c103ffd => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:46:24.939725 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07113017-f398-4eee-bc44-33905a2a4d32 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:24.940015 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07113017-f398-4eee-bc44-33905a2a4d32 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:24.940290 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07113017-f398-4eee-bc44-33905a2a4d32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:24.959666 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-07113017-f398-4eee-bc44-33905a2a4d32 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:46:24.960883 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-07113017-f398-4eee-bc44-33905a2a4d32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:46:25.018555 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1393/2810] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:46:24 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:46:25.028395 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc7fde7-c9ea-4892-a106-f7d3fc6b8c03 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:25.030138 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-abc7fde7-c9ea-4892-a106-f7d3fc6b8c03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:46:25.051830 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:25.088967 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc7fde7-c9ea-4892-a106-f7d3fc6b8c03 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:25.093032 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc80fe3-cbd9-4fff-b85d-78937f52c53a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:25.093662 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc80fe3-cbd9-4fff-b85d-78937f52c53a service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:25.093944 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc80fe3-cbd9-4fff-b85d-78937f52c53a service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:25.094216 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc80fe3-cbd9-4fff-b85d-78937f52c53a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:25.102937 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc80fe3-cbd9-4fff-b85d-78937f52c53a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:25.104367 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc80fe3-cbd9-4fff-b85d-78937f52c53a service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:25.114564 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1418/2811] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:46:25 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:46:29.230232 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b3d807-1963-4bb3-b6f0-5af3aa83936c None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:29.230731 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b3d807-1963-4bb3-b6f0-5af3aa83936c None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:29.230822 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b3d807-1963-4bb3-b6f0-5af3aa83936c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:29.259631 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-d4b3d807-1963-4bb3-b6f0-5af3aa83936c None None] MFA Rules not processed for user `cfa2b12905ac4709947cbeb9db264f33`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:46:29.260848 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-d4b3d807-1963-4bb3-b6f0-5af3aa83936c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:46:29.277808 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:29.336450 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1394/2812] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:46:29 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:46:29.340908 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc80fe3-cbd9-4fff-b85d-78937f52c53a service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:29.351866 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc80fe3-cbd9-4fff-b85d-78937f52c53a service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:29.356864 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b706b8-dfb1-43fa-ba91-d76dd063b054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:29.357726 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b706b8-dfb1-43fa-ba91-d76dd063b054 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:29.358007 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b706b8-dfb1-43fa-ba91-d76dd063b054 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:29.358280 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b706b8-dfb1-43fa-ba91-d76dd063b054 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:29.358931 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b706b8-dfb1-43fa-ba91-d76dd063b054 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:29.361331 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b706b8-dfb1-43fa-ba91-d76dd063b054 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:29.389542 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1419/2813] 10.222.0.102 () {60 vars in 1158 bytes} [Thu Sep 1 17:46:29 2022] POST /identity/v3/projects => generated 374 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:46:29.393287 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b3d807-1963-4bb3-b6f0-5af3aa83936c None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:29.404081 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b3d807-1963-4bb3-b6f0-5af3aa83936c None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:29.462407 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66d643e-54fe-4454-9079-f55e904b1d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:29.463028 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66d643e-54fe-4454-9079-f55e904b1d8b admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:29.463313 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66d643e-54fe-4454-9079-f55e904b1d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:29.463587 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66d643e-54fe-4454-9079-f55e904b1d8b admin admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:29.464228 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66d643e-54fe-4454-9079-f55e904b1d8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:29.466879 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66d643e-54fe-4454-9079-f55e904b1d8b admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:29.514303 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1395/2814] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:46:29 2022] POST /identity/v3/users => generated 294 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:46:29.518276 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b706b8-dfb1-43fa-ba91-d76dd063b054 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:29.528572 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b706b8-dfb1-43fa-ba91-d76dd063b054 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:29.533078 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02abfd10-06c6-44a6-a9d7-15bf439de5cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:29.533769 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02abfd10-06c6-44a6-a9d7-15bf439de5cc admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:29.534046 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02abfd10-06c6-44a6-a9d7-15bf439de5cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:29.534315 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02abfd10-06c6-44a6-a9d7-15bf439de5cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:29.534946 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02abfd10-06c6-44a6-a9d7-15bf439de5cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:29.536512 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02abfd10-06c6-44a6-a9d7-15bf439de5cc admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:29.554667 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1420/2815] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:46:29 2022] GET /identity/v3/roles => generated 1427 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:46:29.558802 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66d643e-54fe-4454-9079-f55e904b1d8b admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:29.570133 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66d643e-54fe-4454-9079-f55e904b1d8b admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:29.574970 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a42c32-d82e-4793-b769-93cf068dc851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:29.575637 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a42c32-d82e-4793-b769-93cf068dc851 admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:29.575922 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a42c32-d82e-4793-b769-93cf068dc851 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:29.576197 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a42c32-d82e-4793-b769-93cf068dc851 admin admin] PATH_INFO: `/v3/projects/f8b7762c0a4b4ebd950b3a6eeb63456f/users/a060151b078e4a299f719fe6fb02d9ef/roles/36577a6f46d642d4b81805ede9872490` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:29.592449 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a42c32-d82e-4793-b769-93cf068dc851 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8b7762c0a4b4ebd950b3a6eeb63456f, user_id=a060151b078e4a299f719fe6fb02d9ef, role_id=36577a6f46d642d4b81805ede9872490)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:29.595218 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a42c32-d82e-4793-b769-93cf068dc851 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:29.610224 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1396/2816] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:46:29 2022] PUT /identity/v3/projects/f8b7762c0a4b4ebd950b3a6eeb63456f/users/a060151b078e4a299f719fe6fb02d9ef/roles/36577a6f46d642d4b81805ede9872490 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:46:29.614095 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02abfd10-06c6-44a6-a9d7-15bf439de5cc admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:29.625642 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02abfd10-06c6-44a6-a9d7-15bf439de5cc admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:29.686966 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41df3668-3acd-4742-a7b7-45253dbc5577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:29.687580 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41df3668-3acd-4742-a7b7-45253dbc5577 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:29.687883 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41df3668-3acd-4742-a7b7-45253dbc5577 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:29.688126 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41df3668-3acd-4742-a7b7-45253dbc5577 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:29.688768 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41df3668-3acd-4742-a7b7-45253dbc5577 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:29.690412 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41df3668-3acd-4742-a7b7-45253dbc5577 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:29.702256 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1421/2817] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:46:29 2022] GET /identity/v3/roles => generated 1427 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:46:29.706389 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a42c32-d82e-4793-b769-93cf068dc851 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:29.716964 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a42c32-d82e-4793-b769-93cf068dc851 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:29.722322 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91af320-fdad-4cd3-a89b-045fdf34c3b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:29.722991 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91af320-fdad-4cd3-a89b-045fdf34c3b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:29.723270 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91af320-fdad-4cd3-a89b-045fdf34c3b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:29.723542 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91af320-fdad-4cd3-a89b-045fdf34c3b1 admin admin] PATH_INFO: `/v3/projects/f8b7762c0a4b4ebd950b3a6eeb63456f/users/a060151b078e4a299f719fe6fb02d9ef/roles/36577a6f46d642d4b81805ede9872490` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:29.725619 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91af320-fdad-4cd3-a89b-045fdf34c3b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8b7762c0a4b4ebd950b3a6eeb63456f, user_id=a060151b078e4a299f719fe6fb02d9ef, role_id=36577a6f46d642d4b81805ede9872490)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:29.728252 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91af320-fdad-4cd3-a89b-045fdf34c3b1 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:29.740155 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1397/2818] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:46:29 2022] PUT /identity/v3/projects/f8b7762c0a4b4ebd950b3a6eeb63456f/users/a060151b078e4a299f719fe6fb02d9ef/roles/36577a6f46d642d4b81805ede9872490 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:46:34.065199 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41df3668-3acd-4742-a7b7-45253dbc5577 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:34.067575 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-41df3668-3acd-4742-a7b7-45253dbc5577 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:46:34.137339 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41df3668-3acd-4742-a7b7-45253dbc5577 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:34.156097 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d889ecc-5a0d-472e-a127-30391e3600f6 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:34.156595 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d889ecc-5a0d-472e-a127-30391e3600f6 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:34.156595 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d889ecc-5a0d-472e-a127-30391e3600f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:34.187470 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:34.191250 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-2d889ecc-5a0d-472e-a127-30391e3600f6 None None] MFA Rules not processed for user `a060151b078e4a299f719fe6fb02d9ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:46:34.192998 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-2d889ecc-5a0d-472e-a127-30391e3600f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:46:34.200381 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f282099-7643-4c92-b239-72ef53bebe27 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '523ed589872a45a09b6a7675a7fb0b75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:34.202130 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f282099-7643-4c92-b239-72ef53bebe27 service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:34.202421 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f282099-7643-4c92-b239-72ef53bebe27 service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:34.202699 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f282099-7643-4c92-b239-72ef53bebe27 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:34.205661 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-3f282099-7643-4c92-b239-72ef53bebe27 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:46:34.268556 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:34.291754 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f282099-7643-4c92-b239-72ef53bebe27 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:34.292257 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1398/2819] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:46:34 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:46:34.293941 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f282099-7643-4c92-b239-72ef53bebe27 service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:34.306441 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d889ecc-5a0d-472e-a127-30391e3600f6 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:34.307373 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1422/2820] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:46:34 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:46:34.308674 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-2d889ecc-5a0d-472e-a127-30391e3600f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:46:34.325142 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d889ecc-5a0d-472e-a127-30391e3600f6 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:34.330876 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf38de84-10f9-4882-b022-384f31ea74e0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f8293316baf4774b62fc11d55d50865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:34.331554 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf38de84-10f9-4882-b022-384f31ea74e0 service nova] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:34.331902 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf38de84-10f9-4882-b022-384f31ea74e0 service nova] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:34.332244 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf38de84-10f9-4882-b022-384f31ea74e0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:34.425112 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf38de84-10f9-4882-b022-384f31ea74e0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:34.426755 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf38de84-10f9-4882-b022-384f31ea74e0 service nova] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:34.441070 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1399/2821] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:46:34 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 17:46:35.598633 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea744447-1bb7-4f3a-ba7f-4d3875bf669d None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:35.599095 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea744447-1bb7-4f3a-ba7f-4d3875bf669d None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:35.599200 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea744447-1bb7-4f3a-ba7f-4d3875bf669d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:35.618153 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-ea744447-1bb7-4f3a-ba7f-4d3875bf669d None None] MFA Rules not processed for user `cfa2b12905ac4709947cbeb9db264f33`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:46:35.619350 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-ea744447-1bb7-4f3a-ba7f-4d3875bf669d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:46:35.626613 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1423/2822] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:46:35 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:46:43.444064 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfac6bce-5f48-4298-aa00-0f3fc7306481 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:43.444590 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfac6bce-5f48-4298-aa00-0f3fc7306481 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:43.444816 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfac6bce-5f48-4298-aa00-0f3fc7306481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:43.466163 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-bfac6bce-5f48-4298-aa00-0f3fc7306481 None None] MFA Rules not processed for user `e7660433575a4daaaed075a6c8934934`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:46:43.467420 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-bfac6bce-5f48-4298-aa00-0f3fc7306481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:46:43.524293 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1400/2823] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:46:43 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:46:43.841792 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea744447-1bb7-4f3a-ba7f-4d3875bf669d None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:43.843829 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-ea744447-1bb7-4f3a-ba7f-4d3875bf669d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:46:43.903851 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea744447-1bb7-4f3a-ba7f-4d3875bf669d None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:43.907970 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866c653b-2d0b-4ebd-883d-e176a34a3427 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:43.908749 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866c653b-2d0b-4ebd-883d-e176a34a3427 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:43.909079 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866c653b-2d0b-4ebd-883d-e176a34a3427 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:43.909378 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866c653b-2d0b-4ebd-883d-e176a34a3427 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:43.911049 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-866c653b-2d0b-4ebd-883d-e176a34a3427 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:46:43.923248 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866c653b-2d0b-4ebd-883d-e176a34a3427 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:43.924767 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866c653b-2d0b-4ebd-883d-e176a34a3427 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:43.933511 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1424/2824] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:46:43 2022] GET /identity/v3/auth/tokens => generated 3131 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:46:44.274201 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfac6bce-5f48-4298-aa00-0f3fc7306481 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:44.284455 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfac6bce-5f48-4298-aa00-0f3fc7306481 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:44.288430 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6443f8ef-f24f-4c1a-9115-06ca84b258fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:44.289026 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6443f8ef-f24f-4c1a-9115-06ca84b258fd service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:44.289324 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6443f8ef-f24f-4c1a-9115-06ca84b258fd service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:44.289615 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6443f8ef-f24f-4c1a-9115-06ca84b258fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:44.291333 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-6443f8ef-f24f-4c1a-9115-06ca84b258fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:46:44.306603 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6443f8ef-f24f-4c1a-9115-06ca84b258fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:44.308396 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6443f8ef-f24f-4c1a-9115-06ca84b258fd service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:44.321957 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1401/2825] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:46:44 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:46:46.271789 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866c653b-2d0b-4ebd-883d-e176a34a3427 service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:46.281800 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866c653b-2d0b-4ebd-883d-e176a34a3427 service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:46.285541 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3bdfdd-1cd8-46f7-89bc-e065e6436722 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:46.286157 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3bdfdd-1cd8-46f7-89bc-e065e6436722 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:46.286437 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3bdfdd-1cd8-46f7-89bc-e065e6436722 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:46.286714 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3bdfdd-1cd8-46f7-89bc-e065e6436722 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:46.288488 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-4b3bdfdd-1cd8-46f7-89bc-e065e6436722 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:46:46.299122 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3bdfdd-1cd8-46f7-89bc-e065e6436722 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:46.300602 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3bdfdd-1cd8-46f7-89bc-e065e6436722 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:46.309529 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1425/2826] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:46:46 2022] GET /identity/v3/auth/tokens => generated 3131 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 17:46:54.315836 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=61 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:56.305677 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=70 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:47:11.849178 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6443f8ef-f24f-4c1a-9115-06ca84b258fd service neutron] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:47:11.862853 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6443f8ef-f24f-4c1a-9115-06ca84b258fd service neutron] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:47:11.868155 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0186ab-a0b8-4d23-bbcc-ea8e63305360 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f8293316baf4774b62fc11d55d50865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:47:11.868777 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0186ab-a0b8-4d23-bbcc-ea8e63305360 service nova] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:11.869061 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0186ab-a0b8-4d23-bbcc-ea8e63305360 service nova] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:11.869442 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0186ab-a0b8-4d23-bbcc-ea8e63305360 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:11.871227 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-6a0186ab-a0b8-4d23-bbcc-ea8e63305360 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:47:11.883657 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0186ab-a0b8-4d23-bbcc-ea8e63305360 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:47:11.885222 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0186ab-a0b8-4d23-bbcc-ea8e63305360 service nova] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:47:11.896688 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1402/2827] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:47:11 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:47:15.214857 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3bdfdd-1cd8-46f7-89bc-e065e6436722 service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:47:15.223455 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3bdfdd-1cd8-46f7-89bc-e065e6436722 service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:47:15.227080 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce2ba60-1470-42f9-8a1b-ff41ec3f0597 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:47:15.227674 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce2ba60-1470-42f9-8a1b-ff41ec3f0597 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:15.227954 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce2ba60-1470-42f9-8a1b-ff41ec3f0597 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:15.228226 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce2ba60-1470-42f9-8a1b-ff41ec3f0597 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:15.237775 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce2ba60-1470-42f9-8a1b-ff41ec3f0597 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:47:15.239181 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce2ba60-1470-42f9-8a1b-ff41ec3f0597 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:47:15.248416 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1426/2828] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:47:15 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:47:21.891700 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=6 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:47:25.244332 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=6 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:48:29.028623 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0186ab-a0b8-4d23-bbcc-ea8e63305360 service nova] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:48:29.041603 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0186ab-a0b8-4d23-bbcc-ea8e63305360 service nova] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:48:29.048695 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb6e6c9-b30a-4afc-ab05-99271c496134 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f8293316baf4774b62fc11d55d50865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:48:29.053353 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb6e6c9-b30a-4afc-ab05-99271c496134 service nova] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:29.053353 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb6e6c9-b30a-4afc-ab05-99271c496134 service nova] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:29.053353 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb6e6c9-b30a-4afc-ab05-99271c496134 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:29.053353 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-dfb6e6c9-b30a-4afc-ab05-99271c496134 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:48:29.069184 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb6e6c9-b30a-4afc-ab05-99271c496134 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:48:29.071419 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb6e6c9-b30a-4afc-ab05-99271c496134 service nova] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:48:29.118592 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1403/2829] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:48:29 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:48:39.099617 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=6 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:48:40.207236 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3173282-5c09-4848-bf1c-b64bf4669201 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:40.207803 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3173282-5c09-4848-bf1c-b64bf4669201 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:40.207803 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3173282-5c09-4848-bf1c-b64bf4669201 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:40.226701 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-f3173282-5c09-4848-bf1c-b64bf4669201 None None] MFA Rules not processed for user `e7660433575a4daaaed075a6c8934934`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:48:40.227921 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-f3173282-5c09-4848-bf1c-b64bf4669201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:48:40.244214 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1427/2830] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:48:40 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:48:50.221179 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:49:59.280340 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb6e6c9-b30a-4afc-ab05-99271c496134 service nova] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:49:59.316777 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb6e6c9-b30a-4afc-ab05-99271c496134 service nova] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:49:59.320479 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2494f4c-f442-4b28-b0aa-c087cee789c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:49:59.321117 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2494f4c-f442-4b28-b0aa-c087cee789c7 service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:49:59.321389 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2494f4c-f442-4b28-b0aa-c087cee789c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:49:59.321660 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2494f4c-f442-4b28-b0aa-c087cee789c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:49:59.323408 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-d2494f4c-f442-4b28-b0aa-c087cee789c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:49:59.356196 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2494f4c-f442-4b28-b0aa-c087cee789c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:49:59.357684 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2494f4c-f442-4b28-b0aa-c087cee789c7 service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:49:59.365683 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1404/2831] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:49:59 2022] GET /identity/v3/auth/tokens => generated 3131 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:49:59.465574 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3173282-5c09-4848-bf1c-b64bf4669201 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:49:59.478218 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3173282-5c09-4848-bf1c-b64bf4669201 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:49:59.481794 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f412b364-3aac-46a3-8714-c140a8afb02e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:49:59.482389 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f412b364-3aac-46a3-8714-c140a8afb02e service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:49:59.482669 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f412b364-3aac-46a3-8714-c140a8afb02e service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:49:59.482949 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f412b364-3aac-46a3-8714-c140a8afb02e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:49:59.484767 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-f412b364-3aac-46a3-8714-c140a8afb02e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:49:59.525459 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f412b364-3aac-46a3-8714-c140a8afb02e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:49:59.526890 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f412b364-3aac-46a3-8714-c140a8afb02e service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:49:59.537773 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1428/2832] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:49:59 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:49:59.603254 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2494f4c-f442-4b28-b0aa-c087cee789c7 service neutron] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:49:59.604607 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-d2494f4c-f442-4b28-b0aa-c087cee789c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:49:59.694453 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2494f4c-f442-4b28-b0aa-c087cee789c7 service neutron] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:49:59.700119 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665bfecd-4712-4191-a5d0-c7d0a584932a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '134ef6039cef468bae0e3f3be79b8073', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:49:59.700721 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665bfecd-4712-4191-a5d0-c7d0a584932a service placement] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:49:59.701007 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665bfecd-4712-4191-a5d0-c7d0a584932a service placement] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:49:59.701282 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665bfecd-4712-4191-a5d0-c7d0a584932a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:49:59.703038 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-665bfecd-4712-4191-a5d0-c7d0a584932a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:49:59.714308 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665bfecd-4712-4191-a5d0-c7d0a584932a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:49:59.715869 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665bfecd-4712-4191-a5d0-c7d0a584932a service placement] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:49:59.725635 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1405/2833] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:49:59 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:50:09.533047 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=20 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:50:09.721394 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=68 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:50:14.678727 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f412b364-3aac-46a3-8714-c140a8afb02e service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:50:14.680189 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-f412b364-3aac-46a3-8714-c140a8afb02e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:50:14.694952 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f412b364-3aac-46a3-8714-c140a8afb02e service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:50:14.701019 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dd7bf7-8e59-4ac0-892a-403d4e726199 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '134ef6039cef468bae0e3f3be79b8073', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:50:14.701781 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dd7bf7-8e59-4ac0-892a-403d4e726199 service placement] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:50:14.702134 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dd7bf7-8e59-4ac0-892a-403d4e726199 service placement] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:50:14.702474 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dd7bf7-8e59-4ac0-892a-403d4e726199 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:50:14.704240 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-51dd7bf7-8e59-4ac0-892a-403d4e726199 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:50:14.716498 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dd7bf7-8e59-4ac0-892a-403d4e726199 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:50:14.718287 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dd7bf7-8e59-4ac0-892a-403d4e726199 service placement] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:50:14.728829 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1429/2834] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:50:14 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:50:20.221957 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665bfecd-4712-4191-a5d0-c7d0a584932a service placement] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:50:20.234515 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665bfecd-4712-4191-a5d0-c7d0a584932a service placement] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:50:20.239871 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cac229c-968d-436c-abfd-3e39050b5ee5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f8293316baf4774b62fc11d55d50865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:50:20.240454 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cac229c-968d-436c-abfd-3e39050b5ee5 service nova] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:50:20.240732 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cac229c-968d-436c-abfd-3e39050b5ee5 service nova] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:50:20.241004 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cac229c-968d-436c-abfd-3e39050b5ee5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:50:20.242975 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-7cac229c-968d-436c-abfd-3e39050b5ee5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:50:20.260283 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cac229c-968d-436c-abfd-3e39050b5ee5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:50:20.261945 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cac229c-968d-436c-abfd-3e39050b5ee5 service nova] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:50:20.271223 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1406/2835] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:50:20 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:50:24.724067 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=6 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:50:30.267113 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=6 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:50:37.783426 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f72939-ed45-4d7a-83e7-1e62869a4323 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:50:37.783930 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f72939-ed45-4d7a-83e7-1e62869a4323 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:50:37.784037 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f72939-ed45-4d7a-83e7-1e62869a4323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:50:37.802935 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-47f72939-ed45-4d7a-83e7-1e62869a4323 None None] MFA Rules not processed for user `e7660433575a4daaaed075a6c8934934`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:50:37.804145 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-47f72939-ed45-4d7a-83e7-1e62869a4323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:50:37.811888 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1430/2836] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:50:37 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:50:47.797279 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:50:50.236881 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cac229c-968d-436c-abfd-3e39050b5ee5 service nova] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:50:50.238514 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-7cac229c-968d-436c-abfd-3e39050b5ee5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:50:50.255532 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cac229c-968d-436c-abfd-3e39050b5ee5 service nova] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:50:50.261600 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d11bfe-1430-4698-9b92-76ec1cf2650f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f8293316baf4774b62fc11d55d50865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:50:50.262295 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d11bfe-1430-4698-9b92-76ec1cf2650f service nova] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:50:50.262578 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d11bfe-1430-4698-9b92-76ec1cf2650f service nova] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:50:50.262860 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d11bfe-1430-4698-9b92-76ec1cf2650f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:50:50.264772 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-96d11bfe-1430-4698-9b92-76ec1cf2650f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:50:50.290961 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d11bfe-1430-4698-9b92-76ec1cf2650f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:50:50.292598 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d11bfe-1430-4698-9b92-76ec1cf2650f service nova] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:50:50.305848 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1407/2837] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:50:50 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:51:00.300293 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=6 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:51:56.785488 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f72939-ed45-4d7a-83e7-1e62869a4323 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:51:56.795322 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f72939-ed45-4d7a-83e7-1e62869a4323 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:51:56.799028 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7ff317-0c0a-42c8-b978-321578d496e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:51:56.799624 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7ff317-0c0a-42c8-b978-321578d496e9 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:51:56.799905 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7ff317-0c0a-42c8-b978-321578d496e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:51:56.800178 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7ff317-0c0a-42c8-b978-321578d496e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:51:56.808383 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7ff317-0c0a-42c8-b978-321578d496e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:51:56.809887 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7ff317-0c0a-42c8-b978-321578d496e9 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:51:56.818862 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1431/2838] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:51:56 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:51:57.752279 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d11bfe-1430-4698-9b92-76ec1cf2650f service nova] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:51:57.763428 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d11bfe-1430-4698-9b92-76ec1cf2650f service nova] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:51:57.767971 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2eff68-29fb-4714-9c94-15ff30bcb785 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:51:57.768618 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2eff68-29fb-4714-9c94-15ff30bcb785 service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:51:57.769060 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2eff68-29fb-4714-9c94-15ff30bcb785 service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:51:57.769412 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2eff68-29fb-4714-9c94-15ff30bcb785 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:51:57.777808 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2eff68-29fb-4714-9c94-15ff30bcb785 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:51:57.780652 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2eff68-29fb-4714-9c94-15ff30bcb785 service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:51:57.791142 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1408/2839] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:51:57 2022] GET /identity/v3/auth/tokens => generated 3131 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:51:58.104804 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7ff317-0c0a-42c8-b978-321578d496e9 service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:51:58.114524 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7ff317-0c0a-42c8-b978-321578d496e9 service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:51:58.118524 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1cc9fc-0ec2-46d9-bf02-2c76d1ac7d2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:51:58.119355 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1cc9fc-0ec2-46d9-bf02-2c76d1ac7d2a service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:51:58.119750 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1cc9fc-0ec2-46d9-bf02-2c76d1ac7d2a service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:51:58.120084 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1cc9fc-0ec2-46d9-bf02-2c76d1ac7d2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:51:58.131600 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1cc9fc-0ec2-46d9-bf02-2c76d1ac7d2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:51:58.133155 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1cc9fc-0ec2-46d9-bf02-2c76d1ac7d2a service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:51:58.156508 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1432/2840] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:51:58 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:51:59.336250 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2eff68-29fb-4714-9c94-15ff30bcb785 service neutron] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:51:59.345594 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2eff68-29fb-4714-9c94-15ff30bcb785 service neutron] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:51:59.361108 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676ba1be-e451-405a-80b6-1b6f90a638c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:51:59.361734 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676ba1be-e451-405a-80b6-1b6f90a638c9 service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:51:59.362007 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676ba1be-e451-405a-80b6-1b6f90a638c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:51:59.362272 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676ba1be-e451-405a-80b6-1b6f90a638c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:51:59.385953 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676ba1be-e451-405a-80b6-1b6f90a638c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:51:59.387442 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676ba1be-e451-405a-80b6-1b6f90a638c9 service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:51:59.400765 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1409/2841] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:51:59 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:52:01.865262 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1cc9fc-0ec2-46d9-bf02-2c76d1ac7d2a service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:52:01.874348 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1cc9fc-0ec2-46d9-bf02-2c76d1ac7d2a service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:52:01.878480 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c6dbaf-930b-4ef7-9350-9f40cd29324b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:52:01.879068 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c6dbaf-930b-4ef7-9350-9f40cd29324b service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:52:01.879516 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c6dbaf-930b-4ef7-9350-9f40cd29324b service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:52:01.879865 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c6dbaf-930b-4ef7-9350-9f40cd29324b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:52:01.950504 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c6dbaf-930b-4ef7-9350-9f40cd29324b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:52:01.951377 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c6dbaf-930b-4ef7-9350-9f40cd29324b service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:52:01.969901 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1433/2842] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:52:01 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:52:05.036485 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676ba1be-e451-405a-80b6-1b6f90a638c9 service neutron] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:52:05.050725 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676ba1be-e451-405a-80b6-1b6f90a638c9 service neutron] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:52:05.058063 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dc8a32-b753-4958-b139-be6e41966d97 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '134ef6039cef468bae0e3f3be79b8073', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:52:05.058684 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dc8a32-b753-4958-b139-be6e41966d97 service placement] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:52:05.058965 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dc8a32-b753-4958-b139-be6e41966d97 service placement] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:52:05.059421 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dc8a32-b753-4958-b139-be6e41966d97 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:52:05.061298 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-37dc8a32-b753-4958-b139-be6e41966d97 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:52:05.075140 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dc8a32-b753-4958-b139-be6e41966d97 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:52:05.076767 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dc8a32-b753-4958-b139-be6e41966d97 service placement] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:52:05.088911 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1410/2843] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:52:05 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:52:08.131154 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c6dbaf-930b-4ef7-9350-9f40cd29324b service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:52:08.141768 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c6dbaf-930b-4ef7-9350-9f40cd29324b service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:52:08.146699 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c17737-9b42-47d2-b141-dc1ade9ac008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:52:08.147467 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c17737-9b42-47d2-b141-dc1ade9ac008 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:52:08.147809 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c17737-9b42-47d2-b141-dc1ade9ac008 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:52:08.148144 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c17737-9b42-47d2-b141-dc1ade9ac008 admin admin] PATH_INFO: `/v3/users/cfa2b12905ac4709947cbeb9db264f33` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:52:08.149425 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c17737-9b42-47d2-b141-dc1ade9ac008 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cfa2b12905ac4709947cbeb9db264f33)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:52:08.151441 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c17737-9b42-47d2-b141-dc1ade9ac008 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:52:08.208811 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b6c17737-9b42-47d2-b141-dc1ade9ac008 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cfa2b12905ac4709947cbeb9db264f33'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:52:08.223013 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b6c17737-9b42-47d2-b141-dc1ade9ac008 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cfa2b12905ac4709947cbeb9db264f33'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:52:08.232135 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b6c17737-9b42-47d2-b141-dc1ade9ac008 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cfa2b12905ac4709947cbeb9db264f33'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:52:08.232734 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b6c17737-9b42-47d2-b141-dc1ade9ac008 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cfa2b12905ac4709947cbeb9db264f33'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:52:08.233385 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b6c17737-9b42-47d2-b141-dc1ade9ac008 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cfa2b12905ac4709947cbeb9db264f33'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:52:08.248331 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1434/2844] 10.222.0.102 () {58 vars in 1229 bytes} [Thu Sep 1 17:52:08 2022] DELETE /identity/v3/users/cfa2b12905ac4709947cbeb9db264f33 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:52:08.425504 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dc8a32-b753-4958-b139-be6e41966d97 service placement] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:52:08.437325 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dc8a32-b753-4958-b139-be6e41966d97 service placement] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:52:08.496422 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa6d3ae-468e-443e-bbd9-dbd68c2632fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:52:08.496992 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa6d3ae-468e-443e-bbd9-dbd68c2632fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:52:08.497274 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa6d3ae-468e-443e-bbd9-dbd68c2632fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:52:08.497642 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa6d3ae-468e-443e-bbd9-dbd68c2632fc admin admin] PATH_INFO: `/v3/projects/85538f245b5b4f4d8799169e2857278d` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:52:08.498834 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa6d3ae-468e-443e-bbd9-dbd68c2632fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=85538f245b5b4f4d8799169e2857278d)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:52:08.500855 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa6d3ae-468e-443e-bbd9-dbd68c2632fc admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:52:08.556927 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-7aa6d3ae-468e-443e-bbd9-dbd68c2632fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85538f245b5b4f4d8799169e2857278d', 'request_id': 'req-7aa6d3ae-468e-443e-bbd9-dbd68c2632fc'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:52:08.563701 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-7aa6d3ae-468e-443e-bbd9-dbd68c2632fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85538f245b5b4f4d8799169e2857278d', 'request_id': 'req-7aa6d3ae-468e-443e-bbd9-dbd68c2632fc'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:52:08.573591 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-7aa6d3ae-468e-443e-bbd9-dbd68c2632fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85538f245b5b4f4d8799169e2857278d', 'request_id': 'req-7aa6d3ae-468e-443e-bbd9-dbd68c2632fc'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:52:08.601914 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-7aa6d3ae-468e-443e-bbd9-dbd68c2632fc admin admin] The token cache is being invalidate because project 85538f245b5b4f4d8799169e2857278d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:52:08.602206 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-7aa6d3ae-468e-443e-bbd9-dbd68c2632fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:52:08.602854 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-7aa6d3ae-468e-443e-bbd9-dbd68c2632fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:52:08.604388 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1411/2845] 10.222.0.102 () {58 vars in 1238 bytes} [Thu Sep 1 17:52:08 2022] DELETE /identity/v3/projects/85538f245b5b4f4d8799169e2857278d => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:52:08.608564 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c17737-9b42-47d2-b141-dc1ade9ac008 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:52:08.610093 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-b6c17737-9b42-47d2-b141-dc1ade9ac008 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:52:08.624082 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c17737-9b42-47d2-b141-dc1ade9ac008 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:52:08.634769 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:52:08.696792 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ca4e57-b411-4143-88b8-d9aaa93f0df1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:52:08.697489 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ca4e57-b411-4143-88b8-d9aaa93f0df1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:52:08.697780 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ca4e57-b411-4143-88b8-d9aaa93f0df1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:52:08.698040 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ca4e57-b411-4143-88b8-d9aaa93f0df1 admin admin] PATH_INFO: `/v3/users/a060151b078e4a299f719fe6fb02d9ef` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:52:08.699248 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ca4e57-b411-4143-88b8-d9aaa93f0df1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a060151b078e4a299f719fe6fb02d9ef)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:52:08.701551 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ca4e57-b411-4143-88b8-d9aaa93f0df1 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:52:08.760351 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b9ca4e57-b411-4143-88b8-d9aaa93f0df1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a060151b078e4a299f719fe6fb02d9ef'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:52:08.777398 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b9ca4e57-b411-4143-88b8-d9aaa93f0df1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a060151b078e4a299f719fe6fb02d9ef'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:52:08.786845 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b9ca4e57-b411-4143-88b8-d9aaa93f0df1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a060151b078e4a299f719fe6fb02d9ef'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:52:08.787534 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b9ca4e57-b411-4143-88b8-d9aaa93f0df1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a060151b078e4a299f719fe6fb02d9ef'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:52:08.788479 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b9ca4e57-b411-4143-88b8-d9aaa93f0df1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a060151b078e4a299f719fe6fb02d9ef'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:52:08.811913 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1435/2846] 10.222.0.102 () {58 vars in 1229 bytes} [Thu Sep 1 17:52:08 2022] DELETE /identity/v3/users/a060151b078e4a299f719fe6fb02d9ef => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:52:09.008401 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa6d3ae-468e-443e-bbd9-dbd68c2632fc admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:52:09.009485 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-7aa6d3ae-468e-443e-bbd9-dbd68c2632fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:52:09.026320 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa6d3ae-468e-443e-bbd9-dbd68c2632fc admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:52:09.066051 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:52:09.086963 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35890e13-1507-49bb-bbf9-519509842840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:52:09.087579 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35890e13-1507-49bb-bbf9-519509842840 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:52:09.087856 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35890e13-1507-49bb-bbf9-519509842840 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:52:09.088127 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35890e13-1507-49bb-bbf9-519509842840 admin admin] PATH_INFO: `/v3/projects/f8b7762c0a4b4ebd950b3a6eeb63456f` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:52:09.089239 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35890e13-1507-49bb-bbf9-519509842840 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8b7762c0a4b4ebd950b3a6eeb63456f)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:52:09.091164 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35890e13-1507-49bb-bbf9-519509842840 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:52:09.136650 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-35890e13-1507-49bb-bbf9-519509842840 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8b7762c0a4b4ebd950b3a6eeb63456f', 'request_id': 'req-35890e13-1507-49bb-bbf9-519509842840'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:52:09.142605 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-35890e13-1507-49bb-bbf9-519509842840 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8b7762c0a4b4ebd950b3a6eeb63456f', 'request_id': 'req-35890e13-1507-49bb-bbf9-519509842840'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:52:09.149587 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-35890e13-1507-49bb-bbf9-519509842840 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8b7762c0a4b4ebd950b3a6eeb63456f', 'request_id': 'req-35890e13-1507-49bb-bbf9-519509842840'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:52:09.160273 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-35890e13-1507-49bb-bbf9-519509842840 admin admin] The token cache is being invalidate because project f8b7762c0a4b4ebd950b3a6eeb63456f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:52:09.160557 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-35890e13-1507-49bb-bbf9-519509842840 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:52:09.161174 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-35890e13-1507-49bb-bbf9-519509842840 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:52:09.162665 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1412/2847] 10.222.0.102 () {58 vars in 1238 bytes} [Thu Sep 1 17:52:09 2022] DELETE /identity/v3/projects/f8b7762c0a4b4ebd950b3a6eeb63456f => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:52:18.807001 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=47,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:52:19.157270 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=33,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:52:28.653730 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ca4e57-b411-4143-88b8-d9aaa93f0df1 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:52:28.655155 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-b9ca4e57-b411-4143-88b8-d9aaa93f0df1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:52:28.716220 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ca4e57-b411-4143-88b8-d9aaa93f0df1 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:52:28.768324 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2f6c04-953c-4c83-b769-ec08caaadbda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:52:28.768937 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2f6c04-953c-4c83-b769-ec08caaadbda service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:52:28.769238 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2f6c04-953c-4c83-b769-ec08caaadbda service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:52:28.769568 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2f6c04-953c-4c83-b769-ec08caaadbda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:52:28.771419 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-aa2f6c04-953c-4c83-b769-ec08caaadbda service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:52:28.837290 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2f6c04-953c-4c83-b769-ec08caaadbda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:52:28.838751 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2f6c04-953c-4c83-b769-ec08caaadbda service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:52:28.848793 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1436/2848] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:52:28 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:52:31.816989 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35890e13-1507-49bb-bbf9-519509842840 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:52:31.818602 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-35890e13-1507-49bb-bbf9-519509842840 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:52:31.882060 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2f6c04-953c-4c83-b769-ec08caaadbda service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:52:31.883510 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-aa2f6c04-953c-4c83-b769-ec08caaadbda service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:52:31.896978 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2f6c04-953c-4c83-b769-ec08caaadbda service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:52:31.902947 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872d3d1b-e8af-40ed-9c15-f009d0d7f0c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:52:31.903670 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872d3d1b-e8af-40ed-9c15-f009d0d7f0c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:52:31.903971 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35890e13-1507-49bb-bbf9-519509842840 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:52:31.904099 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872d3d1b-e8af-40ed-9c15-f009d0d7f0c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:52:31.904405 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872d3d1b-e8af-40ed-9c15-f009d0d7f0c7 admin admin] PATH_INFO: `/v3/users/e7660433575a4daaaed075a6c8934934` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:52:31.905606 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872d3d1b-e8af-40ed-9c15-f009d0d7f0c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7660433575a4daaaed075a6c8934934)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:52:31.907668 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872d3d1b-e8af-40ed-9c15-f009d0d7f0c7 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:52:31.909099 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0f7d42-074c-4364-9cd8-ce00ae5aa21f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '134ef6039cef468bae0e3f3be79b8073', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:52:31.909745 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0f7d42-074c-4364-9cd8-ce00ae5aa21f service placement] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:52:31.910022 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0f7d42-074c-4364-9cd8-ce00ae5aa21f service placement] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:52:31.910355 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0f7d42-074c-4364-9cd8-ce00ae5aa21f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:52:31.912232 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-be0f7d42-074c-4364-9cd8-ce00ae5aa21f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:52:31.925075 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0f7d42-074c-4364-9cd8-ce00ae5aa21f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:52:31.926908 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0f7d42-074c-4364-9cd8-ce00ae5aa21f service placement] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:52:31.939640 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1413/2849] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:52:31 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:52:31.960834 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-872d3d1b-e8af-40ed-9c15-f009d0d7f0c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7660433575a4daaaed075a6c8934934'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:52:31.971950 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:52:31.975751 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-872d3d1b-e8af-40ed-9c15-f009d0d7f0c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7660433575a4daaaed075a6c8934934'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:52:31.983511 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-872d3d1b-e8af-40ed-9c15-f009d0d7f0c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7660433575a4daaaed075a6c8934934'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:52:31.984125 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-872d3d1b-e8af-40ed-9c15-f009d0d7f0c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7660433575a4daaaed075a6c8934934'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:52:31.984712 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-872d3d1b-e8af-40ed-9c15-f009d0d7f0c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7660433575a4daaaed075a6c8934934'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:52:31.997274 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1437/2850] 10.222.0.102 () {58 vars in 1229 bytes} [Thu Sep 1 17:52:31 2022] DELETE /identity/v3/users/e7660433575a4daaaed075a6c8934934 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:52:32.178663 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0f7d42-074c-4364-9cd8-ce00ae5aa21f service placement] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:52:32.191918 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0f7d42-074c-4364-9cd8-ce00ae5aa21f service placement] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:52:32.252561 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d032aee7-e132-44f1-a789-d23f56255fc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:52:32.253100 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d032aee7-e132-44f1-a789-d23f56255fc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:52:32.253484 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d032aee7-e132-44f1-a789-d23f56255fc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:52:32.253761 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d032aee7-e132-44f1-a789-d23f56255fc9 admin admin] PATH_INFO: `/v3/projects/de5f40e77a914f919f5bb15d11181f65` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:52:32.254887 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d032aee7-e132-44f1-a789-d23f56255fc9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=de5f40e77a914f919f5bb15d11181f65)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:52:32.256783 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d032aee7-e132-44f1-a789-d23f56255fc9 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:52:32.305168 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-d032aee7-e132-44f1-a789-d23f56255fc9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de5f40e77a914f919f5bb15d11181f65', 'request_id': 'req-d032aee7-e132-44f1-a789-d23f56255fc9'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:52:32.311436 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-d032aee7-e132-44f1-a789-d23f56255fc9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de5f40e77a914f919f5bb15d11181f65', 'request_id': 'req-d032aee7-e132-44f1-a789-d23f56255fc9'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:52:32.320315 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-d032aee7-e132-44f1-a789-d23f56255fc9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de5f40e77a914f919f5bb15d11181f65', 'request_id': 'req-d032aee7-e132-44f1-a789-d23f56255fc9'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:52:32.332846 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-d032aee7-e132-44f1-a789-d23f56255fc9 admin admin] The token cache is being invalidate because project de5f40e77a914f919f5bb15d11181f65 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:52:32.333201 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-d032aee7-e132-44f1-a789-d23f56255fc9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:52:32.334026 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-d032aee7-e132-44f1-a789-d23f56255fc9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:52:32.336010 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1414/2851] 10.222.0.102 () {58 vars in 1238 bytes} [Thu Sep 1 17:52:32 2022] DELETE /identity/v3/projects/de5f40e77a914f919f5bb15d11181f65 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:52:32.341691 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aa0c4b-4e43-4c72-9079-5b52c73a1887 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:52:32.342036 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aa0c4b-4e43-4c72-9079-5b52c73a1887 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:52:32.342408 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aa0c4b-4e43-4c72-9079-5b52c73a1887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:52:32.403620 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-d1aa0c4b-4e43-4c72-9079-5b52c73a1887 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:52:32.405130 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-d1aa0c4b-4e43-4c72-9079-5b52c73a1887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:52:32.463833 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1438/2852] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:52:32 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:52:32.473092 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129d9a04-fd70-4df1-a3a5-3090a43a5d46 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:52:32.473486 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129d9a04-fd70-4df1-a3a5-3090a43a5d46 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:52:32.473875 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129d9a04-fd70-4df1-a3a5-3090a43a5d46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:52:32.499708 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-129d9a04-fd70-4df1-a3a5-3090a43a5d46 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:52:32.502123 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-129d9a04-fd70-4df1-a3a5-3090a43a5d46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:52:32.514260 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1415/2853] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:52:32 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:52:32.518766 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aa0c4b-4e43-4c72-9079-5b52c73a1887 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:52:32.529298 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aa0c4b-4e43-4c72-9079-5b52c73a1887 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:52:32.533824 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69861eb6-4f6b-4a3c-b9cd-bf1ddf3fb220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:52:32.534599 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69861eb6-4f6b-4a3c-b9cd-bf1ddf3fb220 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:52:32.534881 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69861eb6-4f6b-4a3c-b9cd-bf1ddf3fb220 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:52:32.535150 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69861eb6-4f6b-4a3c-b9cd-bf1ddf3fb220 admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:52:32.535820 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69861eb6-4f6b-4a3c-b9cd-bf1ddf3fb220 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:52:32.536121 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69861eb6-4f6b-4a3c-b9cd-bf1ddf3fb220 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:52:32.537734 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69861eb6-4f6b-4a3c-b9cd-bf1ddf3fb220 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:52:32.549896 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1439/2854] 10.222.0.102 () {58 vars in 1158 bytes} [Thu Sep 1 17:52:32 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:52:32.554169 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129d9a04-fd70-4df1-a3a5-3090a43a5d46 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:52:32.564024 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129d9a04-fd70-4df1-a3a5-3090a43a5d46 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:52:32.568758 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c24f85-1019-4563-a6e5-d151572c2ebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:52:32.569415 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c24f85-1019-4563-a6e5-d151572c2ebb admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:52:32.569694 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c24f85-1019-4563-a6e5-d151572c2ebb admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:52:32.569981 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c24f85-1019-4563-a6e5-d151572c2ebb admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:52:32.570680 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c24f85-1019-4563-a6e5-d151572c2ebb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:52:32.573074 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c24f85-1019-4563-a6e5-d151572c2ebb admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:52:32.584708 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:52:32.602032 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1416/2855] 10.222.0.102 () {60 vars in 1158 bytes} [Thu Sep 1 17:52:32 2022] POST /identity/v3/projects => generated 406 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:52:32.606750 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69861eb6-4f6b-4a3c-b9cd-bf1ddf3fb220 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:52:32.617687 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69861eb6-4f6b-4a3c-b9cd-bf1ddf3fb220 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:52:32.675685 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1306bab-a0a9-4e96-9906-ae56d31856a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:52:32.676290 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1306bab-a0a9-4e96-9906-ae56d31856a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:52:32.676571 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1306bab-a0a9-4e96-9906-ae56d31856a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:52:32.676844 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1306bab-a0a9-4e96-9906-ae56d31856a2 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:52:32.677510 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1306bab-a0a9-4e96-9906-ae56d31856a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:52:32.679455 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1306bab-a0a9-4e96-9906-ae56d31856a2 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:52:32.701130 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1440/2856] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:52:32 2022] POST /identity/v3/users => generated 304 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:52:32.705652 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c24f85-1019-4563-a6e5-d151572c2ebb admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:52:32.716690 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c24f85-1019-4563-a6e5-d151572c2ebb admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:52:32.721863 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b5e101-1406-4944-9971-7cc393bafea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:52:32.722452 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b5e101-1406-4944-9971-7cc393bafea2 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:52:32.722757 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b5e101-1406-4944-9971-7cc393bafea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:52:32.723029 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b5e101-1406-4944-9971-7cc393bafea2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:52:32.723660 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b5e101-1406-4944-9971-7cc393bafea2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:52:32.725260 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b5e101-1406-4944-9971-7cc393bafea2 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:52:32.735146 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1417/2857] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:52:32 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:52:32.739657 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1306bab-a0a9-4e96-9906-ae56d31856a2 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:52:32.749739 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1306bab-a0a9-4e96-9906-ae56d31856a2 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:52:32.754398 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe6bb34-c6cd-4ea5-b9ff-f0c9cbf8135c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:52:32.755014 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe6bb34-c6cd-4ea5-b9ff-f0c9cbf8135c admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:52:32.755269 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe6bb34-c6cd-4ea5-b9ff-f0c9cbf8135c admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:52:32.755543 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe6bb34-c6cd-4ea5-b9ff-f0c9cbf8135c admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:52:32.756175 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe6bb34-c6cd-4ea5-b9ff-f0c9cbf8135c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:52:32.757836 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe6bb34-c6cd-4ea5-b9ff-f0c9cbf8135c admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:52:32.768279 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1441/2858] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:52:32 2022] GET /identity/v3/roles => generated 1427 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:52:32.772262 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b5e101-1406-4944-9971-7cc393bafea2 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:52:32.782665 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b5e101-1406-4944-9971-7cc393bafea2 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:52:32.787231 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3809cc-c9d9-4427-97ff-74d1d0299eb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:52:32.787860 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3809cc-c9d9-4427-97ff-74d1d0299eb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:52:32.788142 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3809cc-c9d9-4427-97ff-74d1d0299eb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:52:32.788413 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3809cc-c9d9-4427-97ff-74d1d0299eb9 admin admin] PATH_INFO: `/v3/projects/a0d21b6a6b0c419f8fe1b8409a896b8a/users/815aa59e30464cc98605acdf9d21d3aa/roles/b75cb360925e4ba5a65f14965c103ffd` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:52:32.803534 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3809cc-c9d9-4427-97ff-74d1d0299eb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0d21b6a6b0c419f8fe1b8409a896b8a, user_id=815aa59e30464cc98605acdf9d21d3aa, role_id=b75cb360925e4ba5a65f14965c103ffd)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:52:32.806175 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3809cc-c9d9-4427-97ff-74d1d0299eb9 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:52:32.819880 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1418/2859] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:52:32 2022] PUT /identity/v3/projects/a0d21b6a6b0c419f8fe1b8409a896b8a/users/815aa59e30464cc98605acdf9d21d3aa/roles/b75cb360925e4ba5a65f14965c103ffd => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:52:32.824861 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ebda74-63f4-4bed-9b07-148d9e8b34df None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:52:32.825172 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ebda74-63f4-4bed-9b07-148d9e8b34df None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:52:32.825481 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ebda74-63f4-4bed-9b07-148d9e8b34df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:52:32.843929 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-03ebda74-63f4-4bed-9b07-148d9e8b34df None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:52:32.845661 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-03ebda74-63f4-4bed-9b07-148d9e8b34df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:52:32.853901 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:52:32.917885 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1442/2860] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:52:32 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:52:32.927845 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3809cc-c9d9-4427-97ff-74d1d0299eb9 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:52:32.929666 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-7d3809cc-c9d9-4427-97ff-74d1d0299eb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:52:32.996398 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3809cc-c9d9-4427-97ff-74d1d0299eb9 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:52:33.000286 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05076e7-493b-43fa-b01b-fd9c5f391c96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:52:33.001445 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05076e7-493b-43fa-b01b-fd9c5f391c96 service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:52:33.001729 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05076e7-493b-43fa-b01b-fd9c5f391c96 service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:52:33.002004 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05076e7-493b-43fa-b01b-fd9c5f391c96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:52:33.011441 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05076e7-493b-43fa-b01b-fd9c5f391c96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:52:33.013005 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05076e7-493b-43fa-b01b-fd9c5f391c96 service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:52:33.024508 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1419/2861] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:52:32 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:52:33.656931 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72407683-aa36-4e08-9491-0038ad483cf3 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:52:33.657815 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72407683-aa36-4e08-9491-0038ad483cf3 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:52:33.658152 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72407683-aa36-4e08-9491-0038ad483cf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:52:33.687089 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-72407683-aa36-4e08-9491-0038ad483cf3 None None] MFA Rules not processed for user `815aa59e30464cc98605acdf9d21d3aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:52:33.688488 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-72407683-aa36-4e08-9491-0038ad483cf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:52:33.759260 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1443/2862] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:52:33 2022] POST /identity/v3/auth/tokens => generated 3292 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:52:33.763579 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05076e7-493b-43fa-b01b-fd9c5f391c96 service neutron] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:52:33.774051 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05076e7-493b-43fa-b01b-fd9c5f391c96 service neutron] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:52:33.778743 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f58929-da14-43fb-a3fc-74348a08cbfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:52:33.779331 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f58929-da14-43fb-a3fc-74348a08cbfb admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:52:33.779610 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f58929-da14-43fb-a3fc-74348a08cbfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:52:33.779884 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f58929-da14-43fb-a3fc-74348a08cbfb admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:52:33.780514 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f58929-da14-43fb-a3fc-74348a08cbfb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:52:33.783583 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f58929-da14-43fb-a3fc-74348a08cbfb admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:52:33.815581 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1420/2863] 10.222.0.102 () {60 vars in 1158 bytes} [Thu Sep 1 17:52:33 2022] POST /identity/v3/projects => generated 404 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:52:33.819910 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72407683-aa36-4e08-9491-0038ad483cf3 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:52:33.831858 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72407683-aa36-4e08-9491-0038ad483cf3 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:52:33.894024 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7422d8c-27d7-4d99-9a45-03ad927e1137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:52:33.894642 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7422d8c-27d7-4d99-9a45-03ad927e1137 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:52:33.894928 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7422d8c-27d7-4d99-9a45-03ad927e1137 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:52:33.895207 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7422d8c-27d7-4d99-9a45-03ad927e1137 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:52:33.895872 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7422d8c-27d7-4d99-9a45-03ad927e1137 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:52:33.897812 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7422d8c-27d7-4d99-9a45-03ad927e1137 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:52:33.919057 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1444/2864] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:52:33 2022] POST /identity/v3/users => generated 309 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:52:33.923446 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f58929-da14-43fb-a3fc-74348a08cbfb admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:52:33.933789 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f58929-da14-43fb-a3fc-74348a08cbfb admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:52:33.938378 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7464c0a4-f3e7-4a8b-a782-96c5c3ba7988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:52:33.938969 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7464c0a4-f3e7-4a8b-a782-96c5c3ba7988 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:52:33.939249 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7464c0a4-f3e7-4a8b-a782-96c5c3ba7988 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:52:33.939522 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7464c0a4-f3e7-4a8b-a782-96c5c3ba7988 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:52:33.940148 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7464c0a4-f3e7-4a8b-a782-96c5c3ba7988 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:52:33.941758 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7464c0a4-f3e7-4a8b-a782-96c5c3ba7988 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:52:33.952099 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1421/2865] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:52:33 2022] GET /identity/v3/roles => generated 1427 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:52:33.956662 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7422d8c-27d7-4d99-9a45-03ad927e1137 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:52:33.966407 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7422d8c-27d7-4d99-9a45-03ad927e1137 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:52:33.971561 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022f33ed-da36-476d-b4e3-b5c047ca0cf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:52:33.972195 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022f33ed-da36-476d-b4e3-b5c047ca0cf7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:52:33.972475 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022f33ed-da36-476d-b4e3-b5c047ca0cf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:52:33.972749 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022f33ed-da36-476d-b4e3-b5c047ca0cf7 admin admin] PATH_INFO: `/v3/projects/76808b6bab824bc5831fe63bfa347ed2/users/5af07885093b4597824d4bc12a705b88/roles/36577a6f46d642d4b81805ede9872490` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:52:33.987588 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022f33ed-da36-476d-b4e3-b5c047ca0cf7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76808b6bab824bc5831fe63bfa347ed2, user_id=5af07885093b4597824d4bc12a705b88, role_id=36577a6f46d642d4b81805ede9872490)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:52:33.990280 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022f33ed-da36-476d-b4e3-b5c047ca0cf7 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:52:34.004147 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1445/2866] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:52:33 2022] PUT /identity/v3/projects/76808b6bab824bc5831fe63bfa347ed2/users/5af07885093b4597824d4bc12a705b88/roles/36577a6f46d642d4b81805ede9872490 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:52:34.008068 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7464c0a4-f3e7-4a8b-a782-96c5c3ba7988 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:52:34.018355 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7464c0a4-f3e7-4a8b-a782-96c5c3ba7988 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:52:34.076545 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd70963-1e57-4d53-8277-fcd55c99cac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:52:34.077168 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd70963-1e57-4d53-8277-fcd55c99cac9 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:52:34.077503 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd70963-1e57-4d53-8277-fcd55c99cac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:52:34.077783 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd70963-1e57-4d53-8277-fcd55c99cac9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:52:34.078416 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd70963-1e57-4d53-8277-fcd55c99cac9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:52:34.080005 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd70963-1e57-4d53-8277-fcd55c99cac9 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:52:34.089793 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1422/2867] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:52:34 2022] GET /identity/v3/roles => generated 1427 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:52:34.093944 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022f33ed-da36-476d-b4e3-b5c047ca0cf7 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:52:34.104130 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022f33ed-da36-476d-b4e3-b5c047ca0cf7 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:52:34.108703 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f249561-559c-4398-bcbd-7f93801421b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:52:34.109439 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f249561-559c-4398-bcbd-7f93801421b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:52:34.109723 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f249561-559c-4398-bcbd-7f93801421b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:52:34.110591 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f249561-559c-4398-bcbd-7f93801421b4 admin admin] PATH_INFO: `/v3/projects/76808b6bab824bc5831fe63bfa347ed2/users/5af07885093b4597824d4bc12a705b88/roles/36577a6f46d642d4b81805ede9872490` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:52:34.112420 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f249561-559c-4398-bcbd-7f93801421b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76808b6bab824bc5831fe63bfa347ed2, user_id=5af07885093b4597824d4bc12a705b88, role_id=36577a6f46d642d4b81805ede9872490)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:52:34.115096 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f249561-559c-4398-bcbd-7f93801421b4 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:52:34.118500 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:52:34.128414 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1446/2868] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:52:34 2022] PUT /identity/v3/projects/76808b6bab824bc5831fe63bfa347ed2/users/5af07885093b4597824d4bc12a705b88/roles/36577a6f46d642d4b81805ede9872490 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:52:35.046984 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af178815-6904-4533-b550-323cd1806b8a None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:52:35.047523 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af178815-6904-4533-b550-323cd1806b8a None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:52:35.048206 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af178815-6904-4533-b550-323cd1806b8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:52:35.076659 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-af178815-6904-4533-b550-323cd1806b8a None None] MFA Rules not processed for user `5af07885093b4597824d4bc12a705b88`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:52:35.078075 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-af178815-6904-4533-b550-323cd1806b8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:52:35.088697 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:52:35.159740 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1423/2869] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:52:35 2022] POST /identity/v3/auth/tokens => generated 3357 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:52:36.289951 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f249561-559c-4398-bcbd-7f93801421b4 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:52:36.354654 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f249561-559c-4398-bcbd-7f93801421b4 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:52:36.359321 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ac0cfe-ae7e-42d7-b04c-6542e5b45078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:52:36.360462 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ac0cfe-ae7e-42d7-b04c-6542e5b45078 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:52:36.360729 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ac0cfe-ae7e-42d7-b04c-6542e5b45078 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:52:36.361002 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ac0cfe-ae7e-42d7-b04c-6542e5b45078 admin admin] PATH_INFO: `/v3/users/815aa59e30464cc98605acdf9d21d3aa` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:52:36.362168 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ac0cfe-ae7e-42d7-b04c-6542e5b45078 admin admin] RBAC: Authorizing `identity:delete_user(user_id=815aa59e30464cc98605acdf9d21d3aa)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:52:36.364137 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ac0cfe-ae7e-42d7-b04c-6542e5b45078 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:52:36.412214 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-03ac0cfe-ae7e-42d7-b04c-6542e5b45078 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '815aa59e30464cc98605acdf9d21d3aa'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:52:36.425590 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-03ac0cfe-ae7e-42d7-b04c-6542e5b45078 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '815aa59e30464cc98605acdf9d21d3aa'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:52:36.433591 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-03ac0cfe-ae7e-42d7-b04c-6542e5b45078 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '815aa59e30464cc98605acdf9d21d3aa'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:52:36.434205 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-03ac0cfe-ae7e-42d7-b04c-6542e5b45078 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '815aa59e30464cc98605acdf9d21d3aa'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:52:36.434808 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-03ac0cfe-ae7e-42d7-b04c-6542e5b45078 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '815aa59e30464cc98605acdf9d21d3aa'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:52:36.447174 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1447/2870] 10.222.0.102 () {58 vars in 1229 bytes} [Thu Sep 1 17:52:36 2022] DELETE /identity/v3/users/815aa59e30464cc98605acdf9d21d3aa => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:52:36.630701 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af178815-6904-4533-b550-323cd1806b8a None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:52:36.642816 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af178815-6904-4533-b550-323cd1806b8a None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:52:36.702925 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8214b7-3baa-44f8-8a6c-5731ba8bfb10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:52:36.703632 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8214b7-3baa-44f8-8a6c-5731ba8bfb10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:52:36.704523 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8214b7-3baa-44f8-8a6c-5731ba8bfb10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:52:36.704804 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8214b7-3baa-44f8-8a6c-5731ba8bfb10 admin admin] PATH_INFO: `/v3/projects/a0d21b6a6b0c419f8fe1b8409a896b8a` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:52:36.706183 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8214b7-3baa-44f8-8a6c-5731ba8bfb10 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0d21b6a6b0c419f8fe1b8409a896b8a)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:52:36.708403 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8214b7-3baa-44f8-8a6c-5731ba8bfb10 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:52:36.759081 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-4c8214b7-3baa-44f8-8a6c-5731ba8bfb10 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0d21b6a6b0c419f8fe1b8409a896b8a', 'request_id': 'req-4c8214b7-3baa-44f8-8a6c-5731ba8bfb10'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:52:36.764537 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-4c8214b7-3baa-44f8-8a6c-5731ba8bfb10 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0d21b6a6b0c419f8fe1b8409a896b8a', 'request_id': 'req-4c8214b7-3baa-44f8-8a6c-5731ba8bfb10'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:52:36.772475 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-4c8214b7-3baa-44f8-8a6c-5731ba8bfb10 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0d21b6a6b0c419f8fe1b8409a896b8a', 'request_id': 'req-4c8214b7-3baa-44f8-8a6c-5731ba8bfb10'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:52:36.784729 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-4c8214b7-3baa-44f8-8a6c-5731ba8bfb10 admin admin] The token cache is being invalidate because project a0d21b6a6b0c419f8fe1b8409a896b8a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:52:36.785015 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-4c8214b7-3baa-44f8-8a6c-5731ba8bfb10 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:52:36.785670 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-4c8214b7-3baa-44f8-8a6c-5731ba8bfb10 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:52:36.787165 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1424/2871] 10.222.0.102 () {58 vars in 1238 bytes} [Thu Sep 1 17:52:36 2022] DELETE /identity/v3/projects/a0d21b6a6b0c419f8fe1b8409a896b8a => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:52:36.791288 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ac0cfe-ae7e-42d7-b04c-6542e5b45078 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:52:36.792694 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-03ac0cfe-ae7e-42d7-b04c-6542e5b45078 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:52:36.806466 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ac0cfe-ae7e-42d7-b04c-6542e5b45078 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:52:36.870650 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2d60dc-2ee0-4c56-91f6-222948943968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:52:36.871293 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2d60dc-2ee0-4c56-91f6-222948943968 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:52:36.871578 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2d60dc-2ee0-4c56-91f6-222948943968 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:52:36.871853 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2d60dc-2ee0-4c56-91f6-222948943968 admin admin] PATH_INFO: `/v3/users/5af07885093b4597824d4bc12a705b88` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:52:36.873064 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2d60dc-2ee0-4c56-91f6-222948943968 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5af07885093b4597824d4bc12a705b88)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:52:36.875116 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2d60dc-2ee0-4c56-91f6-222948943968 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:52:36.891916 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:52:36.926244 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9c2d60dc-2ee0-4c56-91f6-222948943968 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5af07885093b4597824d4bc12a705b88'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:52:36.939613 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9c2d60dc-2ee0-4c56-91f6-222948943968 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5af07885093b4597824d4bc12a705b88'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:52:36.951043 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9c2d60dc-2ee0-4c56-91f6-222948943968 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5af07885093b4597824d4bc12a705b88'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:52:36.951800 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9c2d60dc-2ee0-4c56-91f6-222948943968 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5af07885093b4597824d4bc12a705b88'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:52:36.952644 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9c2d60dc-2ee0-4c56-91f6-222948943968 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5af07885093b4597824d4bc12a705b88'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:52:36.967004 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1448/2872] 10.222.0.102 () {58 vars in 1229 bytes} [Thu Sep 1 17:52:36 2022] DELETE /identity/v3/users/5af07885093b4597824d4bc12a705b88 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:52:37.163766 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8214b7-3baa-44f8-8a6c-5731ba8bfb10 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:52:37.165300 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-4c8214b7-3baa-44f8-8a6c-5731ba8bfb10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:52:37.180597 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8214b7-3baa-44f8-8a6c-5731ba8bfb10 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:52:37.215886 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:52:37.247056 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c322bf4b-e287-445f-8bd3-9eb804b74bb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:52:37.247819 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c322bf4b-e287-445f-8bd3-9eb804b74bb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:52:37.248154 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c322bf4b-e287-445f-8bd3-9eb804b74bb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:52:37.248499 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c322bf4b-e287-445f-8bd3-9eb804b74bb2 admin admin] PATH_INFO: `/v3/projects/76808b6bab824bc5831fe63bfa347ed2` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:52:37.249880 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c322bf4b-e287-445f-8bd3-9eb804b74bb2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=76808b6bab824bc5831fe63bfa347ed2)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:52:37.252466 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c322bf4b-e287-445f-8bd3-9eb804b74bb2 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:52:37.306212 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-c322bf4b-e287-445f-8bd3-9eb804b74bb2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76808b6bab824bc5831fe63bfa347ed2', 'request_id': 'req-c322bf4b-e287-445f-8bd3-9eb804b74bb2'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:52:37.312232 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-c322bf4b-e287-445f-8bd3-9eb804b74bb2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76808b6bab824bc5831fe63bfa347ed2', 'request_id': 'req-c322bf4b-e287-445f-8bd3-9eb804b74bb2'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:52:37.320807 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-c322bf4b-e287-445f-8bd3-9eb804b74bb2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76808b6bab824bc5831fe63bfa347ed2', 'request_id': 'req-c322bf4b-e287-445f-8bd3-9eb804b74bb2'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:52:37.334198 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-c322bf4b-e287-445f-8bd3-9eb804b74bb2 admin admin] The token cache is being invalidate because project 76808b6bab824bc5831fe63bfa347ed2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:52:37.334491 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-c322bf4b-e287-445f-8bd3-9eb804b74bb2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:52:37.335169 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-c322bf4b-e287-445f-8bd3-9eb804b74bb2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:52:37.336796 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1425/2873] 10.222.0.102 () {58 vars in 1238 bytes} [Thu Sep 1 17:52:37 2022] DELETE /identity/v3/projects/76808b6bab824bc5831fe63bfa347ed2 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:52:37.342615 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601ca6f9-c62e-472f-abf4-f2ccea61a3a8 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:52:37.343117 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601ca6f9-c62e-472f-abf4-f2ccea61a3a8 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:52:37.343281 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601ca6f9-c62e-472f-abf4-f2ccea61a3a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:52:37.363777 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-601ca6f9-c62e-472f-abf4-f2ccea61a3a8 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:52:37.365044 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-601ca6f9-c62e-472f-abf4-f2ccea61a3a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:52:37.427554 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1449/2874] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:52:37 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:52:37.435377 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef01d44e-68be-4bd5-8440-8d92f1644e52 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:52:37.435658 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef01d44e-68be-4bd5-8440-8d92f1644e52 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:52:37.435934 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef01d44e-68be-4bd5-8440-8d92f1644e52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:52:37.456842 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-ef01d44e-68be-4bd5-8440-8d92f1644e52 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:52:37.458066 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-ef01d44e-68be-4bd5-8440-8d92f1644e52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:52:37.465975 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1426/2875] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:52:37 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:52:37.470306 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601ca6f9-c62e-472f-abf4-f2ccea61a3a8 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:52:37.479792 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601ca6f9-c62e-472f-abf4-f2ccea61a3a8 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:52:37.484242 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d292d84-291c-4bdd-b03b-352eb133570b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:52:37.484825 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d292d84-291c-4bdd-b03b-352eb133570b admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:52:37.485133 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d292d84-291c-4bdd-b03b-352eb133570b admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:52:37.485448 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d292d84-291c-4bdd-b03b-352eb133570b admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:52:37.486092 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d292d84-291c-4bdd-b03b-352eb133570b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:52:37.486412 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d292d84-291c-4bdd-b03b-352eb133570b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:52:37.488063 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d292d84-291c-4bdd-b03b-352eb133570b admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:52:37.499915 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1450/2876] 10.222.0.102 () {58 vars in 1158 bytes} [Thu Sep 1 17:52:37 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:52:37.504737 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef01d44e-68be-4bd5-8440-8d92f1644e52 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:52:37.514541 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef01d44e-68be-4bd5-8440-8d92f1644e52 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:52:37.520042 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8689c09-cd33-40b3-a873-9d127ddd7bde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:52:37.520717 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8689c09-cd33-40b3-a873-9d127ddd7bde admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:52:37.520917 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8689c09-cd33-40b3-a873-9d127ddd7bde admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:52:37.521249 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8689c09-cd33-40b3-a873-9d127ddd7bde admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:52:37.522037 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8689c09-cd33-40b3-a873-9d127ddd7bde admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:52:37.524039 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8689c09-cd33-40b3-a873-9d127ddd7bde admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:52:37.552054 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1427/2877] 10.222.0.102 () {60 vars in 1158 bytes} [Thu Sep 1 17:52:37 2022] POST /identity/v3/projects => generated 376 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:52:37.556073 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d292d84-291c-4bdd-b03b-352eb133570b admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:52:37.566863 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d292d84-291c-4bdd-b03b-352eb133570b admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:52:37.625031 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6ffedc-f7c0-4899-8531-e017fd9ee753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:52:37.625694 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6ffedc-f7c0-4899-8531-e017fd9ee753 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:52:37.626009 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6ffedc-f7c0-4899-8531-e017fd9ee753 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:52:37.626307 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6ffedc-f7c0-4899-8531-e017fd9ee753 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:52:37.627021 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6ffedc-f7c0-4899-8531-e017fd9ee753 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:52:37.629081 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6ffedc-f7c0-4899-8531-e017fd9ee753 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:52:37.649962 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1451/2878] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:52:37 2022] POST /identity/v3/users => generated 289 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:52:37.654271 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8689c09-cd33-40b3-a873-9d127ddd7bde admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:52:37.665736 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8689c09-cd33-40b3-a873-9d127ddd7bde admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:52:37.670370 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853040ff-47a1-46e0-bbe4-73e45099f5a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:52:37.671000 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853040ff-47a1-46e0-bbe4-73e45099f5a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:52:37.671317 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853040ff-47a1-46e0-bbe4-73e45099f5a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:52:37.671629 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853040ff-47a1-46e0-bbe4-73e45099f5a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:52:37.672412 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853040ff-47a1-46e0-bbe4-73e45099f5a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:52:37.674506 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853040ff-47a1-46e0-bbe4-73e45099f5a4 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:52:37.684841 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1428/2879] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:52:37 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:52:37.689830 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6ffedc-f7c0-4899-8531-e017fd9ee753 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:52:37.702397 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6ffedc-f7c0-4899-8531-e017fd9ee753 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:52:37.707450 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dc8c99-6a1c-49d6-bb73-ebaa94b8d9a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:52:37.708052 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dc8c99-6a1c-49d6-bb73-ebaa94b8d9a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:52:37.708334 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dc8c99-6a1c-49d6-bb73-ebaa94b8d9a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:52:37.708609 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dc8c99-6a1c-49d6-bb73-ebaa94b8d9a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:52:37.709249 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dc8c99-6a1c-49d6-bb73-ebaa94b8d9a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:52:37.710908 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dc8c99-6a1c-49d6-bb73-ebaa94b8d9a4 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:52:37.716921 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:52:37.723039 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1452/2880] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:52:37 2022] GET /identity/v3/roles => generated 1427 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:52:37.727441 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853040ff-47a1-46e0-bbe4-73e45099f5a4 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:52:37.737966 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853040ff-47a1-46e0-bbe4-73e45099f5a4 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:52:37.742814 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aab0a72-6db4-4916-86b4-856dec84164f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:52:37.743455 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aab0a72-6db4-4916-86b4-856dec84164f admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:52:37.743736 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aab0a72-6db4-4916-86b4-856dec84164f admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:52:37.744009 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aab0a72-6db4-4916-86b4-856dec84164f admin admin] PATH_INFO: `/v3/projects/b2bb9e2a6c944afd81d63095fd3a5edd/users/79d749dae7224817a07795db5d206843/roles/b75cb360925e4ba5a65f14965c103ffd` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:52:37.761212 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aab0a72-6db4-4916-86b4-856dec84164f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2bb9e2a6c944afd81d63095fd3a5edd, user_id=79d749dae7224817a07795db5d206843, role_id=b75cb360925e4ba5a65f14965c103ffd)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:52:37.764336 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aab0a72-6db4-4916-86b4-856dec84164f admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:52:37.780898 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1429/2881] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:52:37 2022] PUT /identity/v3/projects/b2bb9e2a6c944afd81d63095fd3a5edd/users/79d749dae7224817a07795db5d206843/roles/b75cb360925e4ba5a65f14965c103ffd => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:52:37.793662 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dfd0a6-e73e-4436-8fc5-b551a07e3d7b None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:52:37.793947 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dfd0a6-e73e-4436-8fc5-b551a07e3d7b None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:52:37.794219 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dfd0a6-e73e-4436-8fc5-b551a07e3d7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:52:37.823897 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-77dfd0a6-e73e-4436-8fc5-b551a07e3d7b None None] MFA Rules not processed for user `79d749dae7224817a07795db5d206843`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:52:37.825112 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-77dfd0a6-e73e-4436-8fc5-b551a07e3d7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:52:37.892824 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1453/2882] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:52:37 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:52:37.897687 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aab0a72-6db4-4916-86b4-856dec84164f admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:52:37.965630 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aab0a72-6db4-4916-86b4-856dec84164f admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:52:37.971360 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83442a8e-cbc3-4b71-b4d1-8bcde18588fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:52:37.972065 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83442a8e-cbc3-4b71-b4d1-8bcde18588fc admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:52:37.972391 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83442a8e-cbc3-4b71-b4d1-8bcde18588fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:52:37.972727 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83442a8e-cbc3-4b71-b4d1-8bcde18588fc admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:52:37.973510 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83442a8e-cbc3-4b71-b4d1-8bcde18588fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:52:37.975756 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83442a8e-cbc3-4b71-b4d1-8bcde18588fc admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:52:37.987391 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:52:38.007404 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1430/2883] 10.222.0.102 () {60 vars in 1158 bytes} [Thu Sep 1 17:52:37 2022] POST /identity/v3/projects => generated 376 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:52:38.011843 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dfd0a6-e73e-4436-8fc5-b551a07e3d7b None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:52:38.022720 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dfd0a6-e73e-4436-8fc5-b551a07e3d7b None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:52:38.079511 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3229d9d4-3098-4437-a708-635780149964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:52:38.080119 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3229d9d4-3098-4437-a708-635780149964 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:52:38.080397 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3229d9d4-3098-4437-a708-635780149964 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:52:38.080665 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3229d9d4-3098-4437-a708-635780149964 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:52:38.081300 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3229d9d4-3098-4437-a708-635780149964 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:52:38.083226 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3229d9d4-3098-4437-a708-635780149964 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:52:38.104544 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1454/2884] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:52:38 2022] POST /identity/v3/users => generated 295 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:52:38.109217 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83442a8e-cbc3-4b71-b4d1-8bcde18588fc admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:52:38.120823 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83442a8e-cbc3-4b71-b4d1-8bcde18588fc admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:52:38.125712 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93ff24f-d121-4258-b66c-944226a493e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:52:38.126304 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93ff24f-d121-4258-b66c-944226a493e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:52:38.126583 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93ff24f-d121-4258-b66c-944226a493e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:52:38.126855 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93ff24f-d121-4258-b66c-944226a493e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:52:38.127511 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93ff24f-d121-4258-b66c-944226a493e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:52:38.129210 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93ff24f-d121-4258-b66c-944226a493e0 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:52:38.139728 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1431/2885] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:52:38 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 17:52:38.144084 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3229d9d4-3098-4437-a708-635780149964 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:52:38.155286 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3229d9d4-3098-4437-a708-635780149964 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:52:38.160626 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6130993-065a-4a3b-8e04-af48058030f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:52:38.161333 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6130993-065a-4a3b-8e04-af48058030f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:52:38.161622 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6130993-065a-4a3b-8e04-af48058030f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:52:38.161973 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6130993-065a-4a3b-8e04-af48058030f2 admin admin] PATH_INFO: `/v3/projects/e8fac82885b047efa1049e05d2fb2ffc/users/a7b28c36453e43ae8c73f5554fe08798/roles/36577a6f46d642d4b81805ede9872490` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:52:38.178467 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6130993-065a-4a3b-8e04-af48058030f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8fac82885b047efa1049e05d2fb2ffc, user_id=a7b28c36453e43ae8c73f5554fe08798, role_id=36577a6f46d642d4b81805ede9872490)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:52:38.181377 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6130993-065a-4a3b-8e04-af48058030f2 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:52:38.194981 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1455/2886] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:52:38 2022] PUT /identity/v3/projects/e8fac82885b047efa1049e05d2fb2ffc/users/a7b28c36453e43ae8c73f5554fe08798/roles/36577a6f46d642d4b81805ede9872490 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:52:38.198940 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93ff24f-d121-4258-b66c-944226a493e0 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:52:38.209515 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93ff24f-d121-4258-b66c-944226a493e0 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:52:38.272614 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1279458e-e48b-4930-b54f-0622def5aaa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:52:38.273133 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1279458e-e48b-4930-b54f-0622def5aaa4 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:52:38.273460 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1279458e-e48b-4930-b54f-0622def5aaa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:52:38.273724 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1279458e-e48b-4930-b54f-0622def5aaa4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:52:38.274362 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1279458e-e48b-4930-b54f-0622def5aaa4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:52:38.275946 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1279458e-e48b-4930-b54f-0622def5aaa4 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:52:38.286788 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1432/2887] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:52:38 2022] GET /identity/v3/roles => generated 1427 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:52:38.293203 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6130993-065a-4a3b-8e04-af48058030f2 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:52:38.309394 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6130993-065a-4a3b-8e04-af48058030f2 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:52:38.313937 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55ead50-e755-4835-80f6-11cd4dbf70cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:52:38.314576 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55ead50-e755-4835-80f6-11cd4dbf70cf admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:52:38.314904 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55ead50-e755-4835-80f6-11cd4dbf70cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:52:38.315181 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55ead50-e755-4835-80f6-11cd4dbf70cf admin admin] PATH_INFO: `/v3/projects/e8fac82885b047efa1049e05d2fb2ffc/users/a7b28c36453e43ae8c73f5554fe08798/roles/36577a6f46d642d4b81805ede9872490` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:52:38.317091 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55ead50-e755-4835-80f6-11cd4dbf70cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8fac82885b047efa1049e05d2fb2ffc, user_id=a7b28c36453e43ae8c73f5554fe08798, role_id=36577a6f46d642d4b81805ede9872490)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:52:38.319772 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55ead50-e755-4835-80f6-11cd4dbf70cf admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:52:38.331000 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1456/2888] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:52:38 2022] PUT /identity/v3/projects/e8fac82885b047efa1049e05d2fb2ffc/users/a7b28c36453e43ae8c73f5554fe08798/roles/36577a6f46d642d4b81805ede9872490 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:52:38.397238 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df26ecb-6d4c-485f-94c0-bb82f4506ce8 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:52:38.397556 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df26ecb-6d4c-485f-94c0-bb82f4506ce8 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:52:38.397840 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df26ecb-6d4c-485f-94c0-bb82f4506ce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:52:38.426880 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-5df26ecb-6d4c-485f-94c0-bb82f4506ce8 None None] MFA Rules not processed for user `a7b28c36453e43ae8c73f5554fe08798`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:52:38.428065 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-5df26ecb-6d4c-485f-94c0-bb82f4506ce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:52:38.497850 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1433/2889] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:52:38 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:52:38.509577 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55ead50-e755-4835-80f6-11cd4dbf70cf admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:52:38.511325 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-c55ead50-e755-4835-80f6-11cd4dbf70cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:52:38.566316 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:52:38.602136 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55ead50-e755-4835-80f6-11cd4dbf70cf admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:52:38.606101 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0457bff-2941-4a2b-b097-d58fcea537ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '523ed589872a45a09b6a7675a7fb0b75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:52:38.606702 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0457bff-2941-4a2b-b097-d58fcea537ea service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:52:38.606993 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0457bff-2941-4a2b-b097-d58fcea537ea service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:52:38.607247 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0457bff-2941-4a2b-b097-d58fcea537ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:52:38.665281 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0457bff-2941-4a2b-b097-d58fcea537ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:52:38.666774 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0457bff-2941-4a2b-b097-d58fcea537ea service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:52:38.676469 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1457/2890] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:52:38 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:52:45.388954 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df26ecb-6d4c-485f-94c0-bb82f4506ce8 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:52:45.400473 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df26ecb-6d4c-485f-94c0-bb82f4506ce8 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:52:45.404107 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635b268e-1866-4354-ad0f-7ee95a808125 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '523ed589872a45a09b6a7675a7fb0b75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:52:45.404702 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635b268e-1866-4354-ad0f-7ee95a808125 service cinder] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:52:45.404985 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635b268e-1866-4354-ad0f-7ee95a808125 service cinder] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:52:45.405252 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635b268e-1866-4354-ad0f-7ee95a808125 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:52:45.414787 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635b268e-1866-4354-ad0f-7ee95a808125 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:52:45.416237 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635b268e-1866-4354-ad0f-7ee95a808125 service cinder] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:52:45.426131 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1434/2891] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:52:45 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:52:48.671705 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=38 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:52:55.421459 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=78,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:53:14.462680 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0457bff-2941-4a2b-b097-d58fcea537ea service cinder] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:53:14.524141 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0457bff-2941-4a2b-b097-d58fcea537ea service cinder] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:53:14.528934 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e6b7bf-567e-4ef6-984d-4dd5f5bb1126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:53:14.529592 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e6b7bf-567e-4ef6-984d-4dd5f5bb1126 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:53:14.529871 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e6b7bf-567e-4ef6-984d-4dd5f5bb1126 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:53:14.530142 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e6b7bf-567e-4ef6-984d-4dd5f5bb1126 admin admin] PATH_INFO: `/v3/users/79d749dae7224817a07795db5d206843` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:53:14.531827 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e6b7bf-567e-4ef6-984d-4dd5f5bb1126 admin admin] RBAC: Authorizing `identity:delete_user(user_id=79d749dae7224817a07795db5d206843)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:53:14.533895 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e6b7bf-567e-4ef6-984d-4dd5f5bb1126 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:53:14.581498 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-85e6b7bf-567e-4ef6-984d-4dd5f5bb1126 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79d749dae7224817a07795db5d206843'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:53:14.593438 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-85e6b7bf-567e-4ef6-984d-4dd5f5bb1126 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79d749dae7224817a07795db5d206843'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:53:14.600832 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-85e6b7bf-567e-4ef6-984d-4dd5f5bb1126 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79d749dae7224817a07795db5d206843'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:53:14.601546 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-85e6b7bf-567e-4ef6-984d-4dd5f5bb1126 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79d749dae7224817a07795db5d206843'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:53:14.602172 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-85e6b7bf-567e-4ef6-984d-4dd5f5bb1126 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79d749dae7224817a07795db5d206843'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:53:14.615186 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1458/2892] 10.222.0.102 () {58 vars in 1229 bytes} [Thu Sep 1 17:53:14 2022] DELETE /identity/v3/users/79d749dae7224817a07795db5d206843 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:53:14.620090 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ab6352-34f4-43bc-afef-479c873254d9 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:53:14.620591 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ab6352-34f4-43bc-afef-479c873254d9 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:53:14.620706 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ab6352-34f4-43bc-afef-479c873254d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:53:14.639819 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-35ab6352-34f4-43bc-afef-479c873254d9 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:53:14.641615 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-35ab6352-34f4-43bc-afef-479c873254d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:53:14.700816 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1435/2893] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:53:14 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:53:14.710625 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e6b7bf-567e-4ef6-984d-4dd5f5bb1126 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:53:14.712002 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-85e6b7bf-567e-4ef6-984d-4dd5f5bb1126 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:53:14.768618 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e6b7bf-567e-4ef6-984d-4dd5f5bb1126 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:53:14.772648 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ceb77c-2dbb-4ddf-99ab-c9a24e37b77c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:53:14.773493 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ceb77c-2dbb-4ddf-99ab-c9a24e37b77c service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:53:14.773885 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ceb77c-2dbb-4ddf-99ab-c9a24e37b77c service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:53:14.774239 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ceb77c-2dbb-4ddf-99ab-c9a24e37b77c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:53:14.783431 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ceb77c-2dbb-4ddf-99ab-c9a24e37b77c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:53:14.784854 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ceb77c-2dbb-4ddf-99ab-c9a24e37b77c service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:53:14.794409 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1459/2894] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:53:14 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:53:15.065890 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ab6352-34f4-43bc-afef-479c873254d9 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:53:15.067825 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-35ab6352-34f4-43bc-afef-479c873254d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:53:15.083549 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ab6352-34f4-43bc-afef-479c873254d9 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:53:15.089196 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b7536a-0d3c-442d-b6bb-b1c3cda06dc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:53:15.089951 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b7536a-0d3c-442d-b6bb-b1c3cda06dc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:53:15.090332 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b7536a-0d3c-442d-b6bb-b1c3cda06dc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:53:15.090707 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b7536a-0d3c-442d-b6bb-b1c3cda06dc5 admin admin] PATH_INFO: `/v3/projects/b2bb9e2a6c944afd81d63095fd3a5edd` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:53:15.092003 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b7536a-0d3c-442d-b6bb-b1c3cda06dc5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2bb9e2a6c944afd81d63095fd3a5edd)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:53:15.094880 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b7536a-0d3c-442d-b6bb-b1c3cda06dc5 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:53:15.146674 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-b2b7536a-0d3c-442d-b6bb-b1c3cda06dc5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2bb9e2a6c944afd81d63095fd3a5edd', 'request_id': 'req-b2b7536a-0d3c-442d-b6bb-b1c3cda06dc5'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:53:15.152556 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-b2b7536a-0d3c-442d-b6bb-b1c3cda06dc5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2bb9e2a6c944afd81d63095fd3a5edd', 'request_id': 'req-b2b7536a-0d3c-442d-b6bb-b1c3cda06dc5'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:53:15.160539 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-b2b7536a-0d3c-442d-b6bb-b1c3cda06dc5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2bb9e2a6c944afd81d63095fd3a5edd', 'request_id': 'req-b2b7536a-0d3c-442d-b6bb-b1c3cda06dc5'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:53:15.172346 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-b2b7536a-0d3c-442d-b6bb-b1c3cda06dc5 admin admin] The token cache is being invalidate because project b2bb9e2a6c944afd81d63095fd3a5edd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:53:15.172635 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-b2b7536a-0d3c-442d-b6bb-b1c3cda06dc5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:53:15.173290 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-b2b7536a-0d3c-442d-b6bb-b1c3cda06dc5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:53:15.174885 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1436/2895] 10.222.0.102 () {58 vars in 1238 bytes} [Thu Sep 1 17:53:15 2022] DELETE /identity/v3/projects/b2bb9e2a6c944afd81d63095fd3a5edd => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:53:15.179493 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ceb77c-2dbb-4ddf-99ab-c9a24e37b77c service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:53:15.189538 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ceb77c-2dbb-4ddf-99ab-c9a24e37b77c service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:53:15.207801 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:53:15.248899 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f6a9f0-9992-4442-a5b7-7df2d10208b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:53:15.250035 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f6a9f0-9992-4442-a5b7-7df2d10208b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:53:15.250308 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f6a9f0-9992-4442-a5b7-7df2d10208b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:53:15.250572 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f6a9f0-9992-4442-a5b7-7df2d10208b0 admin admin] PATH_INFO: `/v3/users/a7b28c36453e43ae8c73f5554fe08798` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:53:15.251682 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f6a9f0-9992-4442-a5b7-7df2d10208b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7b28c36453e43ae8c73f5554fe08798)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:53:15.253787 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f6a9f0-9992-4442-a5b7-7df2d10208b0 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:53:15.302080 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-f1f6a9f0-9992-4442-a5b7-7df2d10208b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7b28c36453e43ae8c73f5554fe08798'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:53:15.315663 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-f1f6a9f0-9992-4442-a5b7-7df2d10208b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7b28c36453e43ae8c73f5554fe08798'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:53:15.323856 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-f1f6a9f0-9992-4442-a5b7-7df2d10208b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7b28c36453e43ae8c73f5554fe08798'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:53:15.324700 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-f1f6a9f0-9992-4442-a5b7-7df2d10208b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7b28c36453e43ae8c73f5554fe08798'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:53:15.325485 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-f1f6a9f0-9992-4442-a5b7-7df2d10208b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7b28c36453e43ae8c73f5554fe08798'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:53:15.339421 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1460/2896] 10.222.0.102 () {58 vars in 1229 bytes} [Thu Sep 1 17:53:15 2022] DELETE /identity/v3/users/a7b28c36453e43ae8c73f5554fe08798 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:53:15.620396 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b7536a-0d3c-442d-b6bb-b1c3cda06dc5 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:53:15.622138 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-b2b7536a-0d3c-442d-b6bb-b1c3cda06dc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:53:15.636926 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b7536a-0d3c-442d-b6bb-b1c3cda06dc5 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:53:15.716815 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb449b3-be02-4ffb-8e3b-8cd78b5c781c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:53:15.717887 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb449b3-be02-4ffb-8e3b-8cd78b5c781c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:53:15.719478 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb449b3-be02-4ffb-8e3b-8cd78b5c781c admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:53:15.719845 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb449b3-be02-4ffb-8e3b-8cd78b5c781c admin admin] PATH_INFO: `/v3/projects/e8fac82885b047efa1049e05d2fb2ffc` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:53:15.721436 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb449b3-be02-4ffb-8e3b-8cd78b5c781c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8fac82885b047efa1049e05d2fb2ffc)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:53:15.723526 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb449b3-be02-4ffb-8e3b-8cd78b5c781c admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:53:15.761993 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:53:15.781353 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-1bb449b3-be02-4ffb-8e3b-8cd78b5c781c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8fac82885b047efa1049e05d2fb2ffc', 'request_id': 'req-1bb449b3-be02-4ffb-8e3b-8cd78b5c781c'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:53:15.786948 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-1bb449b3-be02-4ffb-8e3b-8cd78b5c781c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8fac82885b047efa1049e05d2fb2ffc', 'request_id': 'req-1bb449b3-be02-4ffb-8e3b-8cd78b5c781c'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:53:15.796488 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-1bb449b3-be02-4ffb-8e3b-8cd78b5c781c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8fac82885b047efa1049e05d2fb2ffc', 'request_id': 'req-1bb449b3-be02-4ffb-8e3b-8cd78b5c781c'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:53:15.808923 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-1bb449b3-be02-4ffb-8e3b-8cd78b5c781c admin admin] The token cache is being invalidate because project e8fac82885b047efa1049e05d2fb2ffc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:53:15.809215 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-1bb449b3-be02-4ffb-8e3b-8cd78b5c781c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:53:15.809953 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-1bb449b3-be02-4ffb-8e3b-8cd78b5c781c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:53:15.812229 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1437/2897] 10.222.0.102 () {58 vars in 1238 bytes} [Thu Sep 1 17:53:15 2022] DELETE /identity/v3/projects/e8fac82885b047efa1049e05d2fb2ffc => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:53:15.817520 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8c0f66-aa32-4394-a4ab-a72543ee7f55 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:53:15.818026 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8c0f66-aa32-4394-a4ab-a72543ee7f55 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:53:15.818141 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8c0f66-aa32-4394-a4ab-a72543ee7f55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:53:15.838720 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-2d8c0f66-aa32-4394-a4ab-a72543ee7f55 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:53:15.840023 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-2d8c0f66-aa32-4394-a4ab-a72543ee7f55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:53:15.901052 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1461/2898] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:53:15 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:53:15.909187 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40680011-15be-438f-b069-c992acbafe16 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:53:15.909500 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40680011-15be-438f-b069-c992acbafe16 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:53:15.909772 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40680011-15be-438f-b069-c992acbafe16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:53:15.929819 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-40680011-15be-438f-b069-c992acbafe16 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:53:15.931055 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-40680011-15be-438f-b069-c992acbafe16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:53:15.939340 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1438/2899] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:53:15 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:53:15.943623 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8c0f66-aa32-4394-a4ab-a72543ee7f55 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:53:15.954504 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8c0f66-aa32-4394-a4ab-a72543ee7f55 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:53:15.959443 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062a9b86-6c48-4174-8932-868021d39609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:53:15.960045 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062a9b86-6c48-4174-8932-868021d39609 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:53:15.960354 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062a9b86-6c48-4174-8932-868021d39609 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:53:15.960633 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062a9b86-6c48-4174-8932-868021d39609 admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:53:15.961283 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062a9b86-6c48-4174-8932-868021d39609 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:53:15.961615 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062a9b86-6c48-4174-8932-868021d39609 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:53:15.963212 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062a9b86-6c48-4174-8932-868021d39609 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:53:15.975326 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1462/2900] 10.222.0.102 () {58 vars in 1158 bytes} [Thu Sep 1 17:53:15 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:53:15.979844 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40680011-15be-438f-b069-c992acbafe16 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:53:15.989184 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40680011-15be-438f-b069-c992acbafe16 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:53:15.994108 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d221ca-c76c-4bdb-9c58-fb17063fd3a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:53:15.994698 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d221ca-c76c-4bdb-9c58-fb17063fd3a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:53:15.994980 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d221ca-c76c-4bdb-9c58-fb17063fd3a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:53:15.995251 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d221ca-c76c-4bdb-9c58-fb17063fd3a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:53:15.995971 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d221ca-c76c-4bdb-9c58-fb17063fd3a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:53:15.997839 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d221ca-c76c-4bdb-9c58-fb17063fd3a6 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:53:16.028821 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1439/2901] 10.222.0.102 () {60 vars in 1158 bytes} [Thu Sep 1 17:53:15 2022] POST /identity/v3/projects => generated 368 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:53:16.033038 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062a9b86-6c48-4174-8932-868021d39609 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:53:16.043932 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062a9b86-6c48-4174-8932-868021d39609 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:53:16.071887 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:53:16.107144 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5c92b6-fde0-42c6-b147-0effcb078695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:53:16.107757 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5c92b6-fde0-42c6-b147-0effcb078695 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:53:16.108038 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5c92b6-fde0-42c6-b147-0effcb078695 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:53:16.108329 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5c92b6-fde0-42c6-b147-0effcb078695 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:53:16.108965 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5c92b6-fde0-42c6-b147-0effcb078695 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:53:16.110913 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5c92b6-fde0-42c6-b147-0effcb078695 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:53:16.132504 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1463/2902] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:53:16 2022] POST /identity/v3/users => generated 285 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:53:16.137467 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d221ca-c76c-4bdb-9c58-fb17063fd3a6 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:53:16.149283 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d221ca-c76c-4bdb-9c58-fb17063fd3a6 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:53:16.154107 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39448d9d-b099-4cb2-ac93-f469dbbc1c90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:53:16.154714 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39448d9d-b099-4cb2-ac93-f469dbbc1c90 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:53:16.154992 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39448d9d-b099-4cb2-ac93-f469dbbc1c90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:53:16.155267 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39448d9d-b099-4cb2-ac93-f469dbbc1c90 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:53:16.156479 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39448d9d-b099-4cb2-ac93-f469dbbc1c90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:53:16.158097 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39448d9d-b099-4cb2-ac93-f469dbbc1c90 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:53:16.168727 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1440/2903] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:53:16 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:53:16.173209 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5c92b6-fde0-42c6-b147-0effcb078695 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:53:16.184081 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5c92b6-fde0-42c6-b147-0effcb078695 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:53:16.189518 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0b49e7-d13d-4731-ba70-fea05a43b977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:53:16.190134 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0b49e7-d13d-4731-ba70-fea05a43b977 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:53:16.190415 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0b49e7-d13d-4731-ba70-fea05a43b977 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:53:16.190686 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0b49e7-d13d-4731-ba70-fea05a43b977 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:53:16.191331 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0b49e7-d13d-4731-ba70-fea05a43b977 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:53:16.192928 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0b49e7-d13d-4731-ba70-fea05a43b977 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:53:16.204411 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1464/2904] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:53:16 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:53:16.208828 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39448d9d-b099-4cb2-ac93-f469dbbc1c90 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:53:16.219096 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39448d9d-b099-4cb2-ac93-f469dbbc1c90 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:53:16.224167 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b98065-2313-450b-a511-4a5dcbcd81ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:53:16.224829 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b98065-2313-450b-a511-4a5dcbcd81ae admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:53:16.225110 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b98065-2313-450b-a511-4a5dcbcd81ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:53:16.225416 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b98065-2313-450b-a511-4a5dcbcd81ae admin admin] PATH_INFO: `/v3/projects/6c794a6587854b759de31facf2ee71c6/users/a6f568453169446488a86a279dfb6937/roles/b75cb360925e4ba5a65f14965c103ffd` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:53:16.240297 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b98065-2313-450b-a511-4a5dcbcd81ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c794a6587854b759de31facf2ee71c6, user_id=a6f568453169446488a86a279dfb6937, role_id=b75cb360925e4ba5a65f14965c103ffd)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:53:16.243284 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b98065-2313-450b-a511-4a5dcbcd81ae admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:53:16.257270 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1441/2905] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:53:16 2022] PUT /identity/v3/projects/6c794a6587854b759de31facf2ee71c6/users/a6f568453169446488a86a279dfb6937/roles/b75cb360925e4ba5a65f14965c103ffd => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:53:16.270175 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c750c5a0-608e-4369-8994-5795fd98d60e None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:53:16.270677 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c750c5a0-608e-4369-8994-5795fd98d60e None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:53:16.271081 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c750c5a0-608e-4369-8994-5795fd98d60e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:53:16.299474 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-c750c5a0-608e-4369-8994-5795fd98d60e None None] MFA Rules not processed for user `a6f568453169446488a86a279dfb6937`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:53:16.300690 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-c750c5a0-608e-4369-8994-5795fd98d60e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:53:16.366273 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1465/2906] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:53:16 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:53:16.371404 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b98065-2313-450b-a511-4a5dcbcd81ae admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:53:16.433957 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b98065-2313-450b-a511-4a5dcbcd81ae admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:53:16.443339 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4340cbd2-8554-4f42-99c2-5ddd689f5033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:53:16.443952 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4340cbd2-8554-4f42-99c2-5ddd689f5033 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:53:16.444234 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4340cbd2-8554-4f42-99c2-5ddd689f5033 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:53:16.444510 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4340cbd2-8554-4f42-99c2-5ddd689f5033 admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:53:16.445140 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4340cbd2-8554-4f42-99c2-5ddd689f5033 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:53:16.447090 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4340cbd2-8554-4f42-99c2-5ddd689f5033 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:53:16.474893 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1442/2907] 10.222.0.102 () {60 vars in 1158 bytes} [Thu Sep 1 17:53:16 2022] POST /identity/v3/projects => generated 370 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:53:16.483986 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c750c5a0-608e-4369-8994-5795fd98d60e None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:53:16.491198 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c750c5a0-608e-4369-8994-5795fd98d60e None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:53:16.549330 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bc2f83-ef3a-4c69-bb25-f5bd3f0c563e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:53:16.549969 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bc2f83-ef3a-4c69-bb25-f5bd3f0c563e admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:53:16.550250 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bc2f83-ef3a-4c69-bb25-f5bd3f0c563e admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:53:16.550550 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bc2f83-ef3a-4c69-bb25-f5bd3f0c563e admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:53:16.551218 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bc2f83-ef3a-4c69-bb25-f5bd3f0c563e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:53:16.553165 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bc2f83-ef3a-4c69-bb25-f5bd3f0c563e admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:53:16.575272 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1466/2908] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:53:16 2022] POST /identity/v3/users => generated 292 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:53:16.579995 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4340cbd2-8554-4f42-99c2-5ddd689f5033 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:53:16.590885 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4340cbd2-8554-4f42-99c2-5ddd689f5033 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:53:16.595817 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dad8e1-cb98-45ee-8eff-cd4bdee087e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:53:16.596421 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dad8e1-cb98-45ee-8eff-cd4bdee087e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:53:16.596703 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dad8e1-cb98-45ee-8eff-cd4bdee087e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:53:16.596979 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dad8e1-cb98-45ee-8eff-cd4bdee087e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:53:16.597677 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dad8e1-cb98-45ee-8eff-cd4bdee087e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:53:16.599313 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dad8e1-cb98-45ee-8eff-cd4bdee087e4 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:53:16.610653 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1443/2909] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:53:16 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:53:16.615182 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bc2f83-ef3a-4c69-bb25-f5bd3f0c563e admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:53:16.625499 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bc2f83-ef3a-4c69-bb25-f5bd3f0c563e admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:53:16.630600 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21b6d7a-24b2-4e2d-906c-8fbbc15869ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:53:16.631237 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21b6d7a-24b2-4e2d-906c-8fbbc15869ee admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:53:16.631514 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21b6d7a-24b2-4e2d-906c-8fbbc15869ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:53:16.631782 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21b6d7a-24b2-4e2d-906c-8fbbc15869ee admin admin] PATH_INFO: `/v3/projects/e23bba8e963b48c28d85b9eaf2d80b6d/users/638fdeabddb14ccfa3569bcb2d947bde/roles/36577a6f46d642d4b81805ede9872490` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:53:16.646927 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21b6d7a-24b2-4e2d-906c-8fbbc15869ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=e23bba8e963b48c28d85b9eaf2d80b6d, user_id=638fdeabddb14ccfa3569bcb2d947bde, role_id=36577a6f46d642d4b81805ede9872490)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:53:16.649854 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21b6d7a-24b2-4e2d-906c-8fbbc15869ee admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:53:16.662854 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1467/2910] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:53:16 2022] PUT /identity/v3/projects/e23bba8e963b48c28d85b9eaf2d80b6d/users/638fdeabddb14ccfa3569bcb2d947bde/roles/36577a6f46d642d4b81805ede9872490 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:53:16.666874 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dad8e1-cb98-45ee-8eff-cd4bdee087e4 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:53:16.678297 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dad8e1-cb98-45ee-8eff-cd4bdee087e4 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:53:16.698786 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:53:16.737334 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0333d938-32b8-4a91-8326-394f8c858349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:53:16.737952 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0333d938-32b8-4a91-8326-394f8c858349 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:53:16.738228 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0333d938-32b8-4a91-8326-394f8c858349 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:53:16.738502 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0333d938-32b8-4a91-8326-394f8c858349 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:53:16.739131 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0333d938-32b8-4a91-8326-394f8c858349 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:53:16.740711 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0333d938-32b8-4a91-8326-394f8c858349 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:53:16.751642 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1444/2911] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:53:16 2022] GET /identity/v3/roles => generated 1427 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:53:16.756027 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21b6d7a-24b2-4e2d-906c-8fbbc15869ee admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:53:16.761664 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:53:16.767992 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21b6d7a-24b2-4e2d-906c-8fbbc15869ee admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:53:16.773029 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707f20dc-8a8d-45b7-8a93-1d78859894d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:53:16.773779 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707f20dc-8a8d-45b7-8a93-1d78859894d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:53:16.774059 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707f20dc-8a8d-45b7-8a93-1d78859894d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:53:16.774334 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707f20dc-8a8d-45b7-8a93-1d78859894d8 admin admin] PATH_INFO: `/v3/projects/e23bba8e963b48c28d85b9eaf2d80b6d/users/638fdeabddb14ccfa3569bcb2d947bde/roles/36577a6f46d642d4b81805ede9872490` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:53:16.776261 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707f20dc-8a8d-45b7-8a93-1d78859894d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e23bba8e963b48c28d85b9eaf2d80b6d, user_id=638fdeabddb14ccfa3569bcb2d947bde, role_id=36577a6f46d642d4b81805ede9872490)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:53:16.778989 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707f20dc-8a8d-45b7-8a93-1d78859894d8 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:53:16.789658 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1468/2912] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:53:16 2022] PUT /identity/v3/projects/e23bba8e963b48c28d85b9eaf2d80b6d/users/638fdeabddb14ccfa3569bcb2d947bde/roles/36577a6f46d642d4b81805ede9872490 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:53:16.800548 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ce20a9-cc59-4377-bfde-613e846221f8 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:53:16.800763 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ce20a9-cc59-4377-bfde-613e846221f8 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:53:16.801041 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ce20a9-cc59-4377-bfde-613e846221f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:53:16.829844 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-b2ce20a9-cc59-4377-bfde-613e846221f8 None None] MFA Rules not processed for user `638fdeabddb14ccfa3569bcb2d947bde`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:53:16.831069 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-b2ce20a9-cc59-4377-bfde-613e846221f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:53:16.901914 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1445/2913] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:53:16 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:53:16.913553 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707f20dc-8a8d-45b7-8a93-1d78859894d8 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:53:16.915437 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-707f20dc-8a8d-45b7-8a93-1d78859894d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:53:16.974349 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707f20dc-8a8d-45b7-8a93-1d78859894d8 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:53:16.977987 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb18ccf-1cc9-4c18-8e88-ff1d907bb623 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '523ed589872a45a09b6a7675a7fb0b75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:53:16.978589 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb18ccf-1cc9-4c18-8e88-ff1d907bb623 service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:53:16.978869 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb18ccf-1cc9-4c18-8e88-ff1d907bb623 service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:53:16.979139 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb18ccf-1cc9-4c18-8e88-ff1d907bb623 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:53:16.988983 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb18ccf-1cc9-4c18-8e88-ff1d907bb623 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:53:16.990462 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb18ccf-1cc9-4c18-8e88-ff1d907bb623 service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:53:17.000235 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1469/2914] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:53:16 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:53:17.046941 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ce20a9-cc59-4377-bfde-613e846221f8 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:53:17.048325 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-b2ce20a9-cc59-4377-bfde-613e846221f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:53:17.061088 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ce20a9-cc59-4377-bfde-613e846221f8 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:53:17.064690 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814c78d5-e37b-4a75-895d-e04765a4fe2f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '523ed589872a45a09b6a7675a7fb0b75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:53:17.065344 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814c78d5-e37b-4a75-895d-e04765a4fe2f service cinder] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:53:17.065629 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814c78d5-e37b-4a75-895d-e04765a4fe2f service cinder] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:53:17.065900 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814c78d5-e37b-4a75-895d-e04765a4fe2f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:53:17.124714 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814c78d5-e37b-4a75-895d-e04765a4fe2f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:53:17.126180 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814c78d5-e37b-4a75-895d-e04765a4fe2f service cinder] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:53:17.135124 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1446/2915] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:53:17 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:53:20.530506 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb18ccf-1cc9-4c18-8e88-ff1d907bb623 service cinder] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:53:20.594228 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb18ccf-1cc9-4c18-8e88-ff1d907bb623 service cinder] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:53:20.600323 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206deeb5-0985-4ffc-b110-b6ef4c1231a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:53:20.601433 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206deeb5-0985-4ffc-b110-b6ef4c1231a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:53:20.601766 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206deeb5-0985-4ffc-b110-b6ef4c1231a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:53:20.602164 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206deeb5-0985-4ffc-b110-b6ef4c1231a8 admin admin] PATH_INFO: `/v3/users/a6f568453169446488a86a279dfb6937` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:53:20.603711 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206deeb5-0985-4ffc-b110-b6ef4c1231a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6f568453169446488a86a279dfb6937)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:53:20.606482 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206deeb5-0985-4ffc-b110-b6ef4c1231a8 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:53:20.655821 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-206deeb5-0985-4ffc-b110-b6ef4c1231a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6f568453169446488a86a279dfb6937'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:53:20.667681 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-206deeb5-0985-4ffc-b110-b6ef4c1231a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6f568453169446488a86a279dfb6937'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:53:20.675987 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-206deeb5-0985-4ffc-b110-b6ef4c1231a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6f568453169446488a86a279dfb6937'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:53:20.676685 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-206deeb5-0985-4ffc-b110-b6ef4c1231a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6f568453169446488a86a279dfb6937'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:53:20.677296 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-206deeb5-0985-4ffc-b110-b6ef4c1231a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6f568453169446488a86a279dfb6937'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:53:20.690751 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1470/2916] 10.222.0.102 () {58 vars in 1229 bytes} [Thu Sep 1 17:53:20 2022] DELETE /identity/v3/users/a6f568453169446488a86a279dfb6937 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:53:20.695541 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373db277-823d-422a-802e-47fd7963b18a None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:53:20.696386 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373db277-823d-422a-802e-47fd7963b18a None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:53:20.696654 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373db277-823d-422a-802e-47fd7963b18a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:53:20.718724 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-373db277-823d-422a-802e-47fd7963b18a None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:53:20.719949 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-373db277-823d-422a-802e-47fd7963b18a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:53:20.765476 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:53:20.785674 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1447/2917] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:53:20 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:53:20.796093 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206deeb5-0985-4ffc-b110-b6ef4c1231a8 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:53:20.798197 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-206deeb5-0985-4ffc-b110-b6ef4c1231a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:53:20.817422 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:53:20.874212 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206deeb5-0985-4ffc-b110-b6ef4c1231a8 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:53:20.877919 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71d757a-731b-40a9-b7a2-13c972c98e20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:53:20.878524 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71d757a-731b-40a9-b7a2-13c972c98e20 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:53:20.878803 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71d757a-731b-40a9-b7a2-13c972c98e20 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:53:20.879075 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71d757a-731b-40a9-b7a2-13c972c98e20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:53:20.888275 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71d757a-731b-40a9-b7a2-13c972c98e20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:53:20.889732 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71d757a-731b-40a9-b7a2-13c972c98e20 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:53:20.899247 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1471/2918] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:53:20 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:53:21.193441 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373db277-823d-422a-802e-47fd7963b18a None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:53:21.194949 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-373db277-823d-422a-802e-47fd7963b18a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:53:21.208265 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373db277-823d-422a-802e-47fd7963b18a None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:53:21.213177 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712407da-28ad-4bcd-b9d5-7030da3c7be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:53:21.213969 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712407da-28ad-4bcd-b9d5-7030da3c7be5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:53:21.214320 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712407da-28ad-4bcd-b9d5-7030da3c7be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:53:21.214650 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712407da-28ad-4bcd-b9d5-7030da3c7be5 admin admin] PATH_INFO: `/v3/projects/6c794a6587854b759de31facf2ee71c6` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:53:21.215863 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712407da-28ad-4bcd-b9d5-7030da3c7be5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c794a6587854b759de31facf2ee71c6)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:53:21.218603 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712407da-28ad-4bcd-b9d5-7030da3c7be5 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:53:21.269104 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-712407da-28ad-4bcd-b9d5-7030da3c7be5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c794a6587854b759de31facf2ee71c6', 'request_id': 'req-712407da-28ad-4bcd-b9d5-7030da3c7be5'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:53:21.274942 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-712407da-28ad-4bcd-b9d5-7030da3c7be5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c794a6587854b759de31facf2ee71c6', 'request_id': 'req-712407da-28ad-4bcd-b9d5-7030da3c7be5'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:53:21.283073 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-712407da-28ad-4bcd-b9d5-7030da3c7be5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c794a6587854b759de31facf2ee71c6', 'request_id': 'req-712407da-28ad-4bcd-b9d5-7030da3c7be5'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:53:21.294728 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-712407da-28ad-4bcd-b9d5-7030da3c7be5 admin admin] The token cache is being invalidate because project 6c794a6587854b759de31facf2ee71c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:53:21.295009 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-712407da-28ad-4bcd-b9d5-7030da3c7be5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:53:21.295640 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-712407da-28ad-4bcd-b9d5-7030da3c7be5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:53:21.298670 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1448/2919] 10.222.0.102 () {58 vars in 1238 bytes} [Thu Sep 1 17:53:21 2022] DELETE /identity/v3/projects/6c794a6587854b759de31facf2ee71c6 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:53:21.302747 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71d757a-731b-40a9-b7a2-13c972c98e20 service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:53:21.314352 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71d757a-731b-40a9-b7a2-13c972c98e20 service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:53:21.372584 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b61f4e-93d5-46ff-ab4f-96c33fb07194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:53:21.373267 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b61f4e-93d5-46ff-ab4f-96c33fb07194 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:53:21.373528 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b61f4e-93d5-46ff-ab4f-96c33fb07194 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:53:21.373801 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b61f4e-93d5-46ff-ab4f-96c33fb07194 admin admin] PATH_INFO: `/v3/users/638fdeabddb14ccfa3569bcb2d947bde` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:53:21.375536 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b61f4e-93d5-46ff-ab4f-96c33fb07194 admin admin] RBAC: Authorizing `identity:delete_user(user_id=638fdeabddb14ccfa3569bcb2d947bde)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:53:21.377499 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b61f4e-93d5-46ff-ab4f-96c33fb07194 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:53:21.424913 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-69b61f4e-93d5-46ff-ab4f-96c33fb07194 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '638fdeabddb14ccfa3569bcb2d947bde'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:53:21.437798 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-69b61f4e-93d5-46ff-ab4f-96c33fb07194 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '638fdeabddb14ccfa3569bcb2d947bde'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:53:21.445835 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-69b61f4e-93d5-46ff-ab4f-96c33fb07194 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '638fdeabddb14ccfa3569bcb2d947bde'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:53:21.446445 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-69b61f4e-93d5-46ff-ab4f-96c33fb07194 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '638fdeabddb14ccfa3569bcb2d947bde'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:53:21.447368 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-69b61f4e-93d5-46ff-ab4f-96c33fb07194 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '638fdeabddb14ccfa3569bcb2d947bde'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:53:21.462768 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1472/2920] 10.222.0.102 () {58 vars in 1229 bytes} [Thu Sep 1 17:53:21 2022] DELETE /identity/v3/users/638fdeabddb14ccfa3569bcb2d947bde => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:53:21.759553 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712407da-28ad-4bcd-b9d5-7030da3c7be5 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:53:21.761553 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-712407da-28ad-4bcd-b9d5-7030da3c7be5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:53:21.775415 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712407da-28ad-4bcd-b9d5-7030da3c7be5 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:53:21.846562 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcb1394-67c5-4cc5-8c2f-fdbe028457df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:53:21.847227 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcb1394-67c5-4cc5-8c2f-fdbe028457df admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:53:21.850362 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcb1394-67c5-4cc5-8c2f-fdbe028457df admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:53:21.850362 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcb1394-67c5-4cc5-8c2f-fdbe028457df admin admin] PATH_INFO: `/v3/projects/e23bba8e963b48c28d85b9eaf2d80b6d` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:53:21.850362 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcb1394-67c5-4cc5-8c2f-fdbe028457df admin admin] RBAC: Authorizing `identity:delete_project(project_id=e23bba8e963b48c28d85b9eaf2d80b6d)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:53:21.851332 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcb1394-67c5-4cc5-8c2f-fdbe028457df admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:53:21.907995 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-1bcb1394-67c5-4cc5-8c2f-fdbe028457df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e23bba8e963b48c28d85b9eaf2d80b6d', 'request_id': 'req-1bcb1394-67c5-4cc5-8c2f-fdbe028457df'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:53:21.913476 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-1bcb1394-67c5-4cc5-8c2f-fdbe028457df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e23bba8e963b48c28d85b9eaf2d80b6d', 'request_id': 'req-1bcb1394-67c5-4cc5-8c2f-fdbe028457df'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:53:21.923106 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-1bcb1394-67c5-4cc5-8c2f-fdbe028457df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e23bba8e963b48c28d85b9eaf2d80b6d', 'request_id': 'req-1bcb1394-67c5-4cc5-8c2f-fdbe028457df'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:53:21.931737 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:53:21.937101 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-1bcb1394-67c5-4cc5-8c2f-fdbe028457df admin admin] The token cache is being invalidate because project e23bba8e963b48c28d85b9eaf2d80b6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:53:21.937443 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-1bcb1394-67c5-4cc5-8c2f-fdbe028457df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:53:21.938371 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-1bcb1394-67c5-4cc5-8c2f-fdbe028457df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:53:21.940314 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1449/2921] 10.222.0.102 () {58 vars in 1238 bytes} [Thu Sep 1 17:53:21 2022] DELETE /identity/v3/projects/e23bba8e963b48c28d85b9eaf2d80b6d => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:53:21.945369 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffa6778-fe02-4c27-9c37-987bbfa4f370 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:53:21.946069 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffa6778-fe02-4c27-9c37-987bbfa4f370 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:53:21.946069 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffa6778-fe02-4c27-9c37-987bbfa4f370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:53:21.968718 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-9ffa6778-fe02-4c27-9c37-987bbfa4f370 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:53:21.969970 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-9ffa6778-fe02-4c27-9c37-987bbfa4f370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:53:21.999509 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:53:22.032441 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1473/2922] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:53:21 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:53:22.040590 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5257466-b12a-40e6-8e29-7cced5ccd0bd None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:53:22.041011 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5257466-b12a-40e6-8e29-7cced5ccd0bd None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:53:22.041519 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5257466-b12a-40e6-8e29-7cced5ccd0bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:53:22.061504 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-f5257466-b12a-40e6-8e29-7cced5ccd0bd None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:53:22.062718 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-f5257466-b12a-40e6-8e29-7cced5ccd0bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:53:22.070816 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1450/2923] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:53:22 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:53:22.075349 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffa6778-fe02-4c27-9c37-987bbfa4f370 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:53:22.085717 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffa6778-fe02-4c27-9c37-987bbfa4f370 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:53:22.090253 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde3b6e5-8b93-4a5d-b777-fadd5310e23b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:53:22.090850 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde3b6e5-8b93-4a5d-b777-fadd5310e23b admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:53:22.091131 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde3b6e5-8b93-4a5d-b777-fadd5310e23b admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:53:22.091404 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde3b6e5-8b93-4a5d-b777-fadd5310e23b admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:53:22.092048 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde3b6e5-8b93-4a5d-b777-fadd5310e23b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:53:22.092360 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde3b6e5-8b93-4a5d-b777-fadd5310e23b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:53:22.094599 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde3b6e5-8b93-4a5d-b777-fadd5310e23b admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:53:22.106192 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1474/2924] 10.222.0.102 () {58 vars in 1158 bytes} [Thu Sep 1 17:53:22 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:53:22.111260 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5257466-b12a-40e6-8e29-7cced5ccd0bd None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:53:22.120874 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5257466-b12a-40e6-8e29-7cced5ccd0bd None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:53:22.126018 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2968601-737b-4a7e-bca6-3895ebc55943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:53:22.126607 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2968601-737b-4a7e-bca6-3895ebc55943 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:53:22.126881 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2968601-737b-4a7e-bca6-3895ebc55943 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:53:22.127147 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2968601-737b-4a7e-bca6-3895ebc55943 admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:53:22.127783 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2968601-737b-4a7e-bca6-3895ebc55943 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:53:22.129720 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2968601-737b-4a7e-bca6-3895ebc55943 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:53:22.158982 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1451/2925] 10.222.0.102 () {60 vars in 1158 bytes} [Thu Sep 1 17:53:22 2022] POST /identity/v3/projects => generated 386 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:53:22.162152 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde3b6e5-8b93-4a5d-b777-fadd5310e23b admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:53:22.172721 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde3b6e5-8b93-4a5d-b777-fadd5310e23b admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:53:22.233159 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7012c5db-622c-410a-be5b-7a2925fd5521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:53:22.234371 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7012c5db-622c-410a-be5b-7a2925fd5521 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:53:22.234655 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7012c5db-622c-410a-be5b-7a2925fd5521 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:53:22.234933 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7012c5db-622c-410a-be5b-7a2925fd5521 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:53:22.235593 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7012c5db-622c-410a-be5b-7a2925fd5521 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:53:22.237569 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7012c5db-622c-410a-be5b-7a2925fd5521 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:53:22.260582 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1475/2926] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:53:22 2022] POST /identity/v3/users => generated 294 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:53:22.265039 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2968601-737b-4a7e-bca6-3895ebc55943 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:53:22.275736 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2968601-737b-4a7e-bca6-3895ebc55943 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:53:22.281047 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fa917a-64f8-440f-9e93-1073515581d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:53:22.281707 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fa917a-64f8-440f-9e93-1073515581d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:53:22.281986 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fa917a-64f8-440f-9e93-1073515581d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:53:22.282254 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fa917a-64f8-440f-9e93-1073515581d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:53:22.282918 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fa917a-64f8-440f-9e93-1073515581d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:53:22.284497 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fa917a-64f8-440f-9e93-1073515581d5 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:53:22.295023 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1452/2927] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:53:22 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:53:22.299864 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7012c5db-622c-410a-be5b-7a2925fd5521 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:53:22.310037 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7012c5db-622c-410a-be5b-7a2925fd5521 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:53:22.314800 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d0b2cb-69bc-4fba-af93-8a54c574ab18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:53:22.315401 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d0b2cb-69bc-4fba-af93-8a54c574ab18 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:53:22.315687 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d0b2cb-69bc-4fba-af93-8a54c574ab18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:53:22.315958 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d0b2cb-69bc-4fba-af93-8a54c574ab18 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:53:22.316597 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d0b2cb-69bc-4fba-af93-8a54c574ab18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:53:22.318215 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d0b2cb-69bc-4fba-af93-8a54c574ab18 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:53:22.329292 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1476/2928] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:53:22 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:53:22.334061 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fa917a-64f8-440f-9e93-1073515581d5 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:53:22.345125 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fa917a-64f8-440f-9e93-1073515581d5 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:53:22.350283 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a22045-bcee-47e8-b256-110e12776ba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:53:22.350969 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a22045-bcee-47e8-b256-110e12776ba8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:53:22.351250 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a22045-bcee-47e8-b256-110e12776ba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:53:22.351525 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a22045-bcee-47e8-b256-110e12776ba8 admin admin] PATH_INFO: `/v3/projects/fe25978fc69c4c6a9af4ebb06e821fb2/users/31fcb06129d34343afa78a856f0ab77f/roles/b75cb360925e4ba5a65f14965c103ffd` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:53:22.368652 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a22045-bcee-47e8-b256-110e12776ba8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe25978fc69c4c6a9af4ebb06e821fb2, user_id=31fcb06129d34343afa78a856f0ab77f, role_id=b75cb360925e4ba5a65f14965c103ffd)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:53:22.371361 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a22045-bcee-47e8-b256-110e12776ba8 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:53:22.385096 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1453/2929] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:53:22 2022] PUT /identity/v3/projects/fe25978fc69c4c6a9af4ebb06e821fb2/users/31fcb06129d34343afa78a856f0ab77f/roles/b75cb360925e4ba5a65f14965c103ffd => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:53:22.396926 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a50956-121f-48ac-94a0-b0110e40823d None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:53:22.397234 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a50956-121f-48ac-94a0-b0110e40823d None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:53:22.397556 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a50956-121f-48ac-94a0-b0110e40823d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:53:22.426457 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-21a50956-121f-48ac-94a0-b0110e40823d None None] MFA Rules not processed for user `31fcb06129d34343afa78a856f0ab77f`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:53:22.427592 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-21a50956-121f-48ac-94a0-b0110e40823d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:53:22.491649 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1477/2930] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:53:22 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:53:22.496756 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a22045-bcee-47e8-b256-110e12776ba8 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:53:22.570646 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a22045-bcee-47e8-b256-110e12776ba8 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:53:22.575568 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b59962-7dcb-49a5-999c-4d8c53fa8788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:53:22.576858 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b59962-7dcb-49a5-999c-4d8c53fa8788 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:53:22.576858 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b59962-7dcb-49a5-999c-4d8c53fa8788 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:53:22.576858 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b59962-7dcb-49a5-999c-4d8c53fa8788 admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:53:22.577873 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b59962-7dcb-49a5-999c-4d8c53fa8788 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:53:22.579814 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b59962-7dcb-49a5-999c-4d8c53fa8788 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:53:22.612070 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1454/2931] 10.222.0.102 () {60 vars in 1158 bytes} [Thu Sep 1 17:53:22 2022] POST /identity/v3/projects => generated 384 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:53:22.616655 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a50956-121f-48ac-94a0-b0110e40823d None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:53:22.628553 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a50956-121f-48ac-94a0-b0110e40823d None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:53:22.655322 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:53:22.687276 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c3d4d1-7220-4fe1-9d4a-c7903ed2a8d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:53:22.687800 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c3d4d1-7220-4fe1-9d4a-c7903ed2a8d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:53:22.688079 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c3d4d1-7220-4fe1-9d4a-c7903ed2a8d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:53:22.688352 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c3d4d1-7220-4fe1-9d4a-c7903ed2a8d4 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:53:22.688980 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c3d4d1-7220-4fe1-9d4a-c7903ed2a8d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:53:22.690939 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c3d4d1-7220-4fe1-9d4a-c7903ed2a8d4 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:53:22.712433 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1478/2932] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:53:22 2022] POST /identity/v3/users => generated 299 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:53:22.716661 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b59962-7dcb-49a5-999c-4d8c53fa8788 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:53:22.727677 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b59962-7dcb-49a5-999c-4d8c53fa8788 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:53:22.732298 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e5ede4-212a-4bc5-8a9b-ae400e9d51ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:53:22.732893 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e5ede4-212a-4bc5-8a9b-ae400e9d51ad admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:53:22.733175 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e5ede4-212a-4bc5-8a9b-ae400e9d51ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:53:22.733488 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e5ede4-212a-4bc5-8a9b-ae400e9d51ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:53:22.734124 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e5ede4-212a-4bc5-8a9b-ae400e9d51ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:53:22.735713 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e5ede4-212a-4bc5-8a9b-ae400e9d51ad admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:53:22.746667 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1455/2933] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:53:22 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 17:53:22.751337 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c3d4d1-7220-4fe1-9d4a-c7903ed2a8d4 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:53:22.761852 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c3d4d1-7220-4fe1-9d4a-c7903ed2a8d4 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:53:22.766945 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51ed041-8f2f-43c5-ab68-312cbac7708e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:53:22.767595 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51ed041-8f2f-43c5-ab68-312cbac7708e admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:53:22.767865 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51ed041-8f2f-43c5-ab68-312cbac7708e admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:53:22.768137 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51ed041-8f2f-43c5-ab68-312cbac7708e admin admin] PATH_INFO: `/v3/projects/559761a6089c4a15885f62bd68ce704b/users/05db03604e1a42c6ae2260a21b10f6e6/roles/36577a6f46d642d4b81805ede9872490` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:53:22.784885 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51ed041-8f2f-43c5-ab68-312cbac7708e admin admin] RBAC: Authorizing `identity:create_grant(project_id=559761a6089c4a15885f62bd68ce704b, user_id=05db03604e1a42c6ae2260a21b10f6e6, role_id=36577a6f46d642d4b81805ede9872490)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:53:22.787555 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51ed041-8f2f-43c5-ab68-312cbac7708e admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:53:22.800990 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1479/2934] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:53:22 2022] PUT /identity/v3/projects/559761a6089c4a15885f62bd68ce704b/users/05db03604e1a42c6ae2260a21b10f6e6/roles/36577a6f46d642d4b81805ede9872490 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:53:22.805437 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e5ede4-212a-4bc5-8a9b-ae400e9d51ad admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:53:22.817676 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e5ede4-212a-4bc5-8a9b-ae400e9d51ad admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:53:22.877217 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2995eb18-5dda-47e4-a67d-3030ff04fddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:53:22.877862 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2995eb18-5dda-47e4-a67d-3030ff04fddb admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:53:22.878150 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2995eb18-5dda-47e4-a67d-3030ff04fddb admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:53:22.878426 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2995eb18-5dda-47e4-a67d-3030ff04fddb admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:53:22.879065 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2995eb18-5dda-47e4-a67d-3030ff04fddb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:53:22.880660 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2995eb18-5dda-47e4-a67d-3030ff04fddb admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:53:22.891630 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1456/2935] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:53:22 2022] GET /identity/v3/roles => generated 1427 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:53:22.895797 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51ed041-8f2f-43c5-ab68-312cbac7708e admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:53:22.906696 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51ed041-8f2f-43c5-ab68-312cbac7708e admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:53:22.911510 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae80353-0b37-44ca-9d99-ac93bbe95583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:53:22.912162 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae80353-0b37-44ca-9d99-ac93bbe95583 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:53:22.912446 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae80353-0b37-44ca-9d99-ac93bbe95583 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:53:22.912711 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae80353-0b37-44ca-9d99-ac93bbe95583 admin admin] PATH_INFO: `/v3/projects/559761a6089c4a15885f62bd68ce704b/users/05db03604e1a42c6ae2260a21b10f6e6/roles/36577a6f46d642d4b81805ede9872490` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:53:22.914668 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae80353-0b37-44ca-9d99-ac93bbe95583 admin admin] RBAC: Authorizing `identity:create_grant(project_id=559761a6089c4a15885f62bd68ce704b, user_id=05db03604e1a42c6ae2260a21b10f6e6, role_id=36577a6f46d642d4b81805ede9872490)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:53:22.917314 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae80353-0b37-44ca-9d99-ac93bbe95583 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:53:22.930585 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1480/2936] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:53:22 2022] PUT /identity/v3/projects/559761a6089c4a15885f62bd68ce704b/users/05db03604e1a42c6ae2260a21b10f6e6/roles/36577a6f46d642d4b81805ede9872490 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:53:22.943086 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24199b2-c1fd-47ac-b9ef-7f344ba4513b None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:53:22.943495 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24199b2-c1fd-47ac-b9ef-7f344ba4513b None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:53:22.943984 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24199b2-c1fd-47ac-b9ef-7f344ba4513b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:53:22.956327 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:53:22.982352 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-a24199b2-c1fd-47ac-b9ef-7f344ba4513b None None] MFA Rules not processed for user `05db03604e1a42c6ae2260a21b10f6e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:53:22.983729 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-a24199b2-c1fd-47ac-b9ef-7f344ba4513b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:53:23.053395 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1457/2937] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:53:22 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:53:23.065356 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae80353-0b37-44ca-9d99-ac93bbe95583 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:53:23.067445 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-dae80353-0b37-44ca-9d99-ac93bbe95583 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:53:23.127020 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae80353-0b37-44ca-9d99-ac93bbe95583 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:53:23.130715 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4de4d5-a4e5-4da8-99ab-7fdce956cd41 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '523ed589872a45a09b6a7675a7fb0b75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:53:23.131315 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4de4d5-a4e5-4da8-99ab-7fdce956cd41 service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:53:23.131591 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4de4d5-a4e5-4da8-99ab-7fdce956cd41 service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:53:23.131861 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4de4d5-a4e5-4da8-99ab-7fdce956cd41 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:53:23.189957 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4de4d5-a4e5-4da8-99ab-7fdce956cd41 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:53:23.191393 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4de4d5-a4e5-4da8-99ab-7fdce956cd41 service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:53:23.200282 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1481/2938] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:53:23 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:53:27.463737 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24199b2-c1fd-47ac-b9ef-7f344ba4513b None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:53:27.465164 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-a24199b2-c1fd-47ac-b9ef-7f344ba4513b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:53:27.477020 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24199b2-c1fd-47ac-b9ef-7f344ba4513b None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:53:27.480523 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b58360-78ac-4216-9983-dc8ae799e6cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '523ed589872a45a09b6a7675a7fb0b75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:53:27.481108 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b58360-78ac-4216-9983-dc8ae799e6cd service cinder] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:53:27.481422 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b58360-78ac-4216-9983-dc8ae799e6cd service cinder] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:53:27.481692 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b58360-78ac-4216-9983-dc8ae799e6cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:53:27.490390 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b58360-78ac-4216-9983-dc8ae799e6cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:53:27.491804 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b58360-78ac-4216-9983-dc8ae799e6cd service cinder] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:53:27.501959 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1458/2939] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:53:27 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:53:32.414565 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4de4d5-a4e5-4da8-99ab-7fdce956cd41 service cinder] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:53:32.453790 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:53:32.478133 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4de4d5-a4e5-4da8-99ab-7fdce956cd41 service cinder] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:53:32.482686 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a506fea5-12ac-464d-81be-a642a47ea14e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:53:32.483826 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a506fea5-12ac-464d-81be-a642a47ea14e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:53:32.484102 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a506fea5-12ac-464d-81be-a642a47ea14e admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:53:32.484376 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a506fea5-12ac-464d-81be-a642a47ea14e admin admin] PATH_INFO: `/v3/users/31fcb06129d34343afa78a856f0ab77f` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:53:32.485523 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a506fea5-12ac-464d-81be-a642a47ea14e admin admin] RBAC: Authorizing `identity:delete_user(user_id=31fcb06129d34343afa78a856f0ab77f)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:53:32.487462 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a506fea5-12ac-464d-81be-a642a47ea14e admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:53:32.535568 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-a506fea5-12ac-464d-81be-a642a47ea14e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31fcb06129d34343afa78a856f0ab77f'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:53:32.547349 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-a506fea5-12ac-464d-81be-a642a47ea14e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31fcb06129d34343afa78a856f0ab77f'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:53:32.554862 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-a506fea5-12ac-464d-81be-a642a47ea14e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31fcb06129d34343afa78a856f0ab77f'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:53:32.555474 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-a506fea5-12ac-464d-81be-a642a47ea14e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31fcb06129d34343afa78a856f0ab77f'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:53:32.556129 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-a506fea5-12ac-464d-81be-a642a47ea14e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31fcb06129d34343afa78a856f0ab77f'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:53:32.568284 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1482/2940] 10.222.0.102 () {58 vars in 1229 bytes} [Thu Sep 1 17:53:32 2022] DELETE /identity/v3/users/31fcb06129d34343afa78a856f0ab77f => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:53:32.572998 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b35b9dc-37f1-4f2a-8d12-a6d2339a8552 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:53:32.573536 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b35b9dc-37f1-4f2a-8d12-a6d2339a8552 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:53:32.573656 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b35b9dc-37f1-4f2a-8d12-a6d2339a8552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:53:32.594700 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-2b35b9dc-37f1-4f2a-8d12-a6d2339a8552 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:53:32.595940 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-2b35b9dc-37f1-4f2a-8d12-a6d2339a8552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:53:32.656397 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1459/2941] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:53:32 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:53:32.666367 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a506fea5-12ac-464d-81be-a642a47ea14e admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:53:32.667764 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-a506fea5-12ac-464d-81be-a642a47ea14e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:53:32.724722 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a506fea5-12ac-464d-81be-a642a47ea14e admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:53:32.728359 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6711bf87-17c4-4a9f-85bf-f64b9412b7be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:53:32.728959 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6711bf87-17c4-4a9f-85bf-f64b9412b7be service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:53:32.729238 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6711bf87-17c4-4a9f-85bf-f64b9412b7be service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:53:32.729545 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6711bf87-17c4-4a9f-85bf-f64b9412b7be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:53:32.738446 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6711bf87-17c4-4a9f-85bf-f64b9412b7be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:53:32.739868 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6711bf87-17c4-4a9f-85bf-f64b9412b7be service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:53:32.750024 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1483/2942] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:53:32 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:53:33.015223 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b35b9dc-37f1-4f2a-8d12-a6d2339a8552 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:53:33.016699 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-2b35b9dc-37f1-4f2a-8d12-a6d2339a8552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:53:33.029036 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b35b9dc-37f1-4f2a-8d12-a6d2339a8552 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:53:33.034554 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b57e6ee-da3b-4a27-a634-48a97bfe8ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:53:33.035483 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b57e6ee-da3b-4a27-a634-48a97bfe8ecc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:53:33.035811 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b57e6ee-da3b-4a27-a634-48a97bfe8ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:53:33.036367 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b57e6ee-da3b-4a27-a634-48a97bfe8ecc admin admin] PATH_INFO: `/v3/projects/fe25978fc69c4c6a9af4ebb06e821fb2` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:53:33.038498 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b57e6ee-da3b-4a27-a634-48a97bfe8ecc admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe25978fc69c4c6a9af4ebb06e821fb2)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:53:33.040466 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b57e6ee-da3b-4a27-a634-48a97bfe8ecc admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:53:33.090161 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-2b57e6ee-da3b-4a27-a634-48a97bfe8ecc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe25978fc69c4c6a9af4ebb06e821fb2', 'request_id': 'req-2b57e6ee-da3b-4a27-a634-48a97bfe8ecc'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:53:33.096386 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-2b57e6ee-da3b-4a27-a634-48a97bfe8ecc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe25978fc69c4c6a9af4ebb06e821fb2', 'request_id': 'req-2b57e6ee-da3b-4a27-a634-48a97bfe8ecc'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:53:33.103826 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-2b57e6ee-da3b-4a27-a634-48a97bfe8ecc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe25978fc69c4c6a9af4ebb06e821fb2', 'request_id': 'req-2b57e6ee-da3b-4a27-a634-48a97bfe8ecc'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:53:33.112111 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:53:33.116254 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-2b57e6ee-da3b-4a27-a634-48a97bfe8ecc admin admin] The token cache is being invalidate because project fe25978fc69c4c6a9af4ebb06e821fb2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:53:33.117123 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-2b57e6ee-da3b-4a27-a634-48a97bfe8ecc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:53:33.117916 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-2b57e6ee-da3b-4a27-a634-48a97bfe8ecc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:53:33.119446 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1460/2943] 10.222.0.102 () {58 vars in 1238 bytes} [Thu Sep 1 17:53:33 2022] DELETE /identity/v3/projects/fe25978fc69c4c6a9af4ebb06e821fb2 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:53:33.123976 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6711bf87-17c4-4a9f-85bf-f64b9412b7be service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:53:33.134075 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6711bf87-17c4-4a9f-85bf-f64b9412b7be service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:53:33.192326 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:53:33.204009 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01dc753-1932-40c9-91f1-16c310800c1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:53:33.204652 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01dc753-1932-40c9-91f1-16c310800c1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:53:33.204970 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01dc753-1932-40c9-91f1-16c310800c1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:53:33.205244 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01dc753-1932-40c9-91f1-16c310800c1f admin admin] PATH_INFO: `/v3/users/05db03604e1a42c6ae2260a21b10f6e6` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:53:33.206521 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01dc753-1932-40c9-91f1-16c310800c1f admin admin] RBAC: Authorizing `identity:delete_user(user_id=05db03604e1a42c6ae2260a21b10f6e6)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:53:33.208475 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01dc753-1932-40c9-91f1-16c310800c1f admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:53:33.253607 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-d01dc753-1932-40c9-91f1-16c310800c1f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05db03604e1a42c6ae2260a21b10f6e6'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:53:33.266217 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-d01dc753-1932-40c9-91f1-16c310800c1f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05db03604e1a42c6ae2260a21b10f6e6'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:53:33.273231 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-d01dc753-1932-40c9-91f1-16c310800c1f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05db03604e1a42c6ae2260a21b10f6e6'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:53:33.273861 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-d01dc753-1932-40c9-91f1-16c310800c1f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05db03604e1a42c6ae2260a21b10f6e6'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:53:33.274461 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-d01dc753-1932-40c9-91f1-16c310800c1f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05db03604e1a42c6ae2260a21b10f6e6'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:53:33.287203 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1484/2944] 10.222.0.102 () {58 vars in 1229 bytes} [Thu Sep 1 17:53:33 2022] DELETE /identity/v3/users/05db03604e1a42c6ae2260a21b10f6e6 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:53:33.572202 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b57e6ee-da3b-4a27-a634-48a97bfe8ecc admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:53:33.574210 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-2b57e6ee-da3b-4a27-a634-48a97bfe8ecc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:53:33.589593 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b57e6ee-da3b-4a27-a634-48a97bfe8ecc admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:53:33.651843 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefd32c1-a486-4b76-b937-6fd69320f913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:53:33.652452 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefd32c1-a486-4b76-b937-6fd69320f913 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:53:33.652734 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefd32c1-a486-4b76-b937-6fd69320f913 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:53:33.653035 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefd32c1-a486-4b76-b937-6fd69320f913 admin admin] PATH_INFO: `/v3/projects/559761a6089c4a15885f62bd68ce704b` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:53:33.654228 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefd32c1-a486-4b76-b937-6fd69320f913 admin admin] RBAC: Authorizing `identity:delete_project(project_id=559761a6089c4a15885f62bd68ce704b)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:53:33.656190 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefd32c1-a486-4b76-b937-6fd69320f913 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:53:33.701156 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-fefd32c1-a486-4b76-b937-6fd69320f913 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '559761a6089c4a15885f62bd68ce704b', 'request_id': 'req-fefd32c1-a486-4b76-b937-6fd69320f913'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:53:33.707069 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-fefd32c1-a486-4b76-b937-6fd69320f913 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '559761a6089c4a15885f62bd68ce704b', 'request_id': 'req-fefd32c1-a486-4b76-b937-6fd69320f913'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:53:33.713749 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-fefd32c1-a486-4b76-b937-6fd69320f913 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '559761a6089c4a15885f62bd68ce704b', 'request_id': 'req-fefd32c1-a486-4b76-b937-6fd69320f913'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:53:33.725178 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-fefd32c1-a486-4b76-b937-6fd69320f913 admin admin] The token cache is being invalidate because project 559761a6089c4a15885f62bd68ce704b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:53:33.725500 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-fefd32c1-a486-4b76-b937-6fd69320f913 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:53:33.726140 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-fefd32c1-a486-4b76-b937-6fd69320f913 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:53:33.727616 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1461/2945] 10.222.0.102 () {58 vars in 1238 bytes} [Thu Sep 1 17:53:33 2022] DELETE /identity/v3/projects/559761a6089c4a15885f62bd68ce704b => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:53:33.734808 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f708f2-718f-4b55-b407-ae2456055267 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:53:33.735297 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f708f2-718f-4b55-b407-ae2456055267 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:53:33.735420 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f708f2-718f-4b55-b407-ae2456055267 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:53:33.759183 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-77f708f2-718f-4b55-b407-ae2456055267 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:53:33.760396 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-77f708f2-718f-4b55-b407-ae2456055267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:53:33.820763 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1485/2946] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:53:33 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:53:33.830968 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9639cd09-3a86-426e-a293-2aa80ee95be2 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:53:33.831255 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9639cd09-3a86-426e-a293-2aa80ee95be2 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:53:33.831527 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9639cd09-3a86-426e-a293-2aa80ee95be2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:53:33.851861 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-9639cd09-3a86-426e-a293-2aa80ee95be2 None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:53:33.853094 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-9639cd09-3a86-426e-a293-2aa80ee95be2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:53:33.861391 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1462/2947] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:53:33 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:53:33.865610 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f708f2-718f-4b55-b407-ae2456055267 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:53:33.878293 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f708f2-718f-4b55-b407-ae2456055267 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:53:33.883734 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fafd86-a15c-4044-821d-dea8d8351c13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:53:33.884384 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fafd86-a15c-4044-821d-dea8d8351c13 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:53:33.884718 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fafd86-a15c-4044-821d-dea8d8351c13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:53:33.885056 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fafd86-a15c-4044-821d-dea8d8351c13 admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:53:33.886003 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fafd86-a15c-4044-821d-dea8d8351c13 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:53:33.886364 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fafd86-a15c-4044-821d-dea8d8351c13 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:53:33.888760 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fafd86-a15c-4044-821d-dea8d8351c13 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:53:33.901714 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1486/2948] 10.222.0.102 () {58 vars in 1158 bytes} [Thu Sep 1 17:53:33 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:53:33.906820 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9639cd09-3a86-426e-a293-2aa80ee95be2 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:53:33.916269 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9639cd09-3a86-426e-a293-2aa80ee95be2 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:53:33.921204 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8379db85-f3cb-4546-bc3c-5375b414c277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:53:33.921858 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8379db85-f3cb-4546-bc3c-5375b414c277 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:53:33.922141 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8379db85-f3cb-4546-bc3c-5375b414c277 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:53:33.922418 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8379db85-f3cb-4546-bc3c-5375b414c277 admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:53:33.923062 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8379db85-f3cb-4546-bc3c-5375b414c277 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:53:33.924966 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8379db85-f3cb-4546-bc3c-5375b414c277 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:53:33.953289 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1463/2949] 10.222.0.102 () {60 vars in 1158 bytes} [Thu Sep 1 17:53:33 2022] POST /identity/v3/projects => generated 376 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:53:33.957507 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fafd86-a15c-4044-821d-dea8d8351c13 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:53:33.968309 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fafd86-a15c-4044-821d-dea8d8351c13 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:53:34.025192 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd8eb3d-792c-4000-95e1-c093910016ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:53:34.025837 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd8eb3d-792c-4000-95e1-c093910016ed admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:53:34.026119 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd8eb3d-792c-4000-95e1-c093910016ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:53:34.026396 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd8eb3d-792c-4000-95e1-c093910016ed admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:53:34.027028 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd8eb3d-792c-4000-95e1-c093910016ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:53:34.029066 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd8eb3d-792c-4000-95e1-c093910016ed admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:53:34.039752 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:53:34.056495 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1487/2950] 10.222.0.102 () {60 vars in 1149 bytes} [Thu Sep 1 17:53:33 2022] POST /identity/v3/users => generated 289 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:53:34.057707 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8379db85-f3cb-4546-bc3c-5375b414c277 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:53:34.068110 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8379db85-f3cb-4546-bc3c-5375b414c277 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:53:34.072643 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0f3df9-a856-45f1-babd-9d5d84ed4c20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:53:34.073229 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0f3df9-a856-45f1-babd-9d5d84ed4c20 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:53:34.073545 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0f3df9-a856-45f1-babd-9d5d84ed4c20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:53:34.073813 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0f3df9-a856-45f1-babd-9d5d84ed4c20 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:53:34.074449 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0f3df9-a856-45f1-babd-9d5d84ed4c20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:53:34.076053 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0f3df9-a856-45f1-babd-9d5d84ed4c20 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:53:34.086602 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1464/2951] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:53:34 2022] GET /identity/v3/roles => generated 1427 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:53:34.090970 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd8eb3d-792c-4000-95e1-c093910016ed admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:53:34.104699 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd8eb3d-792c-4000-95e1-c093910016ed admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:53:34.111292 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca758c1-347b-4f04-8704-c1608703d632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:53:34.112029 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca758c1-347b-4f04-8704-c1608703d632 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:53:34.112421 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca758c1-347b-4f04-8704-c1608703d632 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:53:34.112866 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca758c1-347b-4f04-8704-c1608703d632 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:53:34.113784 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca758c1-347b-4f04-8704-c1608703d632 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:53:34.115802 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca758c1-347b-4f04-8704-c1608703d632 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:53:34.131918 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1488/2952] 10.222.0.102 () {58 vars in 1127 bytes} [Thu Sep 1 17:53:34 2022] GET /identity/v3/roles => generated 1427 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:53:34.135988 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0f3df9-a856-45f1-babd-9d5d84ed4c20 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:53:34.146054 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0f3df9-a856-45f1-babd-9d5d84ed4c20 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:53:34.150707 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebe7d33-098a-4295-b3c3-a704ea9bf1b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:53:34.151369 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebe7d33-098a-4295-b3c3-a704ea9bf1b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:53:34.151655 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebe7d33-098a-4295-b3c3-a704ea9bf1b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:53:34.151938 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebe7d33-098a-4295-b3c3-a704ea9bf1b3 admin admin] PATH_INFO: `/v3/projects/d7fd7779e2fe4451b8c687ebe56636bb/users/30013966a1594740939e3c68f6c5d5e2/roles/b75cb360925e4ba5a65f14965c103ffd` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:53:34.167310 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebe7d33-098a-4295-b3c3-a704ea9bf1b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7fd7779e2fe4451b8c687ebe56636bb, user_id=30013966a1594740939e3c68f6c5d5e2, role_id=b75cb360925e4ba5a65f14965c103ffd)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:53:34.170031 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebe7d33-098a-4295-b3c3-a704ea9bf1b3 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:53:34.184035 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1465/2953] 10.222.0.102 () {60 vars in 1488 bytes} [Thu Sep 1 17:53:34 2022] PUT /identity/v3/projects/d7fd7779e2fe4451b8c687ebe56636bb/users/30013966a1594740939e3c68f6c5d5e2/roles/b75cb360925e4ba5a65f14965c103ffd => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:53:34.196110 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dc059a-2103-4744-a953-3d0b7f54e70f None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:53:34.196376 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dc059a-2103-4744-a953-3d0b7f54e70f None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:53:34.196654 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dc059a-2103-4744-a953-3d0b7f54e70f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:53:34.226221 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-20dc059a-2103-4744-a953-3d0b7f54e70f None None] MFA Rules not processed for user `30013966a1594740939e3c68f6c5d5e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:53:34.227386 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-20dc059a-2103-4744-a953-3d0b7f54e70f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:53:34.293141 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1489/2954] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:53:34 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:53:34.305684 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebe7d33-098a-4295-b3c3-a704ea9bf1b3 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:53:34.307383 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-aebe7d33-098a-4295-b3c3-a704ea9bf1b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:53:34.340558 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:53:34.372156 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebe7d33-098a-4295-b3c3-a704ea9bf1b3 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:53:34.376313 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e3bc81-f794-4f5d-8d27-9b4752a7eca5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '523ed589872a45a09b6a7675a7fb0b75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:53:34.377122 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e3bc81-f794-4f5d-8d27-9b4752a7eca5 service cinder] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:53:34.377436 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e3bc81-f794-4f5d-8d27-9b4752a7eca5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:53:34.377709 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e3bc81-f794-4f5d-8d27-9b4752a7eca5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:53:34.386453 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e3bc81-f794-4f5d-8d27-9b4752a7eca5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:53:34.387871 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e3bc81-f794-4f5d-8d27-9b4752a7eca5 service cinder] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:53:34.396560 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1466/2955] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:53:34 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:53:44.290111 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:INSERT=1,keystone:SELECT=39 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:53:44.393097 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=14 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:53:56.100823 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dc059a-2103-4744-a953-3d0b7f54e70f None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:53:56.162939 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dc059a-2103-4744-a953-3d0b7f54e70f None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:53:56.167411 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977a6eeb-cc05-412c-ae23-e0d4b32bd6e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:53:56.168036 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977a6eeb-cc05-412c-ae23-e0d4b32bd6e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:53:56.168321 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977a6eeb-cc05-412c-ae23-e0d4b32bd6e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:53:56.168596 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977a6eeb-cc05-412c-ae23-e0d4b32bd6e5 admin admin] PATH_INFO: `/v3/users/30013966a1594740939e3c68f6c5d5e2` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:53:56.169744 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977a6eeb-cc05-412c-ae23-e0d4b32bd6e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=30013966a1594740939e3c68f6c5d5e2)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:53:56.171662 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977a6eeb-cc05-412c-ae23-e0d4b32bd6e5 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:53:56.217004 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-977a6eeb-cc05-412c-ae23-e0d4b32bd6e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30013966a1594740939e3c68f6c5d5e2'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:53:56.229872 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-977a6eeb-cc05-412c-ae23-e0d4b32bd6e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30013966a1594740939e3c68f6c5d5e2'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:53:56.237823 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-977a6eeb-cc05-412c-ae23-e0d4b32bd6e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30013966a1594740939e3c68f6c5d5e2'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:53:56.238459 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-977a6eeb-cc05-412c-ae23-e0d4b32bd6e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30013966a1594740939e3c68f6c5d5e2'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:53:56.239086 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-977a6eeb-cc05-412c-ae23-e0d4b32bd6e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30013966a1594740939e3c68f6c5d5e2'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:53:56.251355 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1490/2956] 10.222.0.102 () {58 vars in 1229 bytes} [Thu Sep 1 17:53:56 2022] DELETE /identity/v3/users/30013966a1594740939e3c68f6c5d5e2 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:53:56.255833 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b06ab0-17a3-4fda-b9fe-23c63cca387a None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:53:56.256320 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b06ab0-17a3-4fda-b9fe-23c63cca387a None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:53:56.256454 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b06ab0-17a3-4fda-b9fe-23c63cca387a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:53:56.275738 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-54b06ab0-17a3-4fda-b9fe-23c63cca387a None None] MFA Rules not processed for user `e6c9197e0f844432a03b9e75fc87712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:53:56.276924 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-54b06ab0-17a3-4fda-b9fe-23c63cca387a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:53:56.335389 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1467/2957] 10.222.0.102 () {58 vars in 963 bytes} [Thu Sep 1 17:53:56 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:53:56.344985 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977a6eeb-cc05-412c-ae23-e0d4b32bd6e5 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:53:56.346887 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-977a6eeb-cc05-412c-ae23-e0d4b32bd6e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:53:56.405711 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977a6eeb-cc05-412c-ae23-e0d4b32bd6e5 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:53:56.410172 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376fd02f-17d8-4013-807e-8ff9c94a4455 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c7d3594fdab4191be1d841af43a5c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:53:56.410830 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376fd02f-17d8-4013-807e-8ff9c94a4455 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:53:56.411113 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376fd02f-17d8-4013-807e-8ff9c94a4455 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:53:56.411388 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376fd02f-17d8-4013-807e-8ff9c94a4455 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:53:56.421334 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376fd02f-17d8-4013-807e-8ff9c94a4455 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:53:56.422781 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376fd02f-17d8-4013-807e-8ff9c94a4455 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:53:56.432275 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1491/2958] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:53:56 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:53:56.694232 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b06ab0-17a3-4fda-b9fe-23c63cca387a None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:53:56.695723 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-54b06ab0-17a3-4fda-b9fe-23c63cca387a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:53:56.708669 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b06ab0-17a3-4fda-b9fe-23c63cca387a None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:53:56.713277 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1cca6c-d86d-41b1-8250-6781a994253b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c9197e0f844432a03b9e75fc87712b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ace9b3845813477eb192c158600d0beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:53:56.713944 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1cca6c-d86d-41b1-8250-6781a994253b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:53:56.714225 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1cca6c-d86d-41b1-8250-6781a994253b admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:53:56.714606 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1cca6c-d86d-41b1-8250-6781a994253b admin admin] PATH_INFO: `/v3/projects/d7fd7779e2fe4451b8c687ebe56636bb` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:53:56.715851 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1cca6c-d86d-41b1-8250-6781a994253b admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7fd7779e2fe4451b8c687ebe56636bb)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:53:56.717798 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1cca6c-d86d-41b1-8250-6781a994253b admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:53:56.764162 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-8a1cca6c-d86d-41b1-8250-6781a994253b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7fd7779e2fe4451b8c687ebe56636bb', 'request_id': 'req-8a1cca6c-d86d-41b1-8250-6781a994253b'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:53:56.770290 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-8a1cca6c-d86d-41b1-8250-6781a994253b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7fd7779e2fe4451b8c687ebe56636bb', 'request_id': 'req-8a1cca6c-d86d-41b1-8250-6781a994253b'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:53:56.778603 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-8a1cca6c-d86d-41b1-8250-6781a994253b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7fd7779e2fe4451b8c687ebe56636bb', 'request_id': 'req-8a1cca6c-d86d-41b1-8250-6781a994253b'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:53:56.790266 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-8a1cca6c-d86d-41b1-8250-6781a994253b admin admin] The token cache is being invalidate because project d7fd7779e2fe4451b8c687ebe56636bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:53:56.790551 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-8a1cca6c-d86d-41b1-8250-6781a994253b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:53:56.791172 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-8a1cca6c-d86d-41b1-8250-6781a994253b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:53:56.792625 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1468/2959] 10.222.0.102 () {58 vars in 1238 bytes} [Thu Sep 1 17:53:56 2022] DELETE /identity/v3/projects/d7fd7779e2fe4451b8c687ebe56636bb => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:54:06.428157 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:54:06.786805 dv-cinder-852211-8-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:55:18.664551 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376fd02f-17d8-4013-807e-8ff9c94a4455 service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:55:18.666143 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-376fd02f-17d8-4013-807e-8ff9c94a4455 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:55:18.737202 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376fd02f-17d8-4013-807e-8ff9c94a4455 service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:55:18.795757 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb0b223-645b-4485-9b47-b8b2cb1c827a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '134ef6039cef468bae0e3f3be79b8073', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78fff7f1f6e4f68b8d233ad2e5bbd4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:55:18.796357 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb0b223-645b-4485-9b47-b8b2cb1c827a service placement] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:55:18.796637 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb0b223-645b-4485-9b47-b8b2cb1c827a service placement] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:55:18.796907 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb0b223-645b-4485-9b47-b8b2cb1c827a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:55:18.798645 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-aeb0b223-645b-4485-9b47-b8b2cb1c827a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:55:18.810550 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb0b223-645b-4485-9b47-b8b2cb1c827a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:55:18.812067 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb0b223-645b-4485-9b47-b8b2cb1c827a service placement] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:55:18.825963 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1492/2960] 10.222.0.102 () {60 vars in 1370 bytes} [Thu Sep 1 17:55:18 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:55:28.821703 dv-cinder-852211-8-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=58 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}